-- Logs begin at Tue 2022-05-17 18:27:01 UTC, end at Tue 2022-05-17 19:57:47 UTC. -- May 17 19:09:08.096530 dv-cinder-840678-10-iscsi systemd[1]: Starting Devstack devstack@keystone.service... May 17 19:09:08.109363 dv-cinder-840678-10-iscsi devstack@keystone.service[80151]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini May 17 19:09:08.111337 dv-cinder-840678-10-iscsi devstack@keystone.service[80151]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue May 17 19:09:08 2022] *** May 17 19:09:08.111337 dv-cinder-840678-10-iscsi devstack@keystone.service[80151]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 May 17 19:09:08.111337 dv-cinder-840678-10-iscsi devstack@keystone.service[80151]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 May 17 19:09:08.111337 dv-cinder-840678-10-iscsi devstack@keystone.service[80151]: nodename: dv-cinder-840678-10-iscsi May 17 19:09:08.111337 dv-cinder-840678-10-iscsi devstack@keystone.service[80151]: machine: x86_64 May 17 19:09:08.111337 dv-cinder-840678-10-iscsi devstack@keystone.service[80151]: clock source: unix May 17 19:09:08.111337 dv-cinder-840678-10-iscsi devstack@keystone.service[80151]: pcre jit disabled May 17 19:09:08.111337 dv-cinder-840678-10-iscsi devstack@keystone.service[80151]: detected number of CPU cores: 4 May 17 19:09:08.111337 dv-cinder-840678-10-iscsi devstack@keystone.service[80151]: current working directory: / May 17 19:09:08.111337 dv-cinder-840678-10-iscsi devstack@keystone.service[80151]: detected binary path: /usr/bin/uwsgi-core May 17 19:09:08.111337 dv-cinder-840678-10-iscsi devstack@keystone.service[80151]: your processes number limit is 47680 May 17 19:09:08.111337 dv-cinder-840678-10-iscsi devstack@keystone.service[80151]: your memory page size is 4096 bytes May 17 19:09:08.111337 dv-cinder-840678-10-iscsi devstack@keystone.service[80151]: detected max file descriptor number: 2048 May 17 19:09:08.111337 dv-cinder-840678-10-iscsi devstack@keystone.service[80151]: lock engine: pthread robust mutexes May 17 19:09:08.112150 dv-cinder-840678-10-iscsi devstack@keystone.service[80151]: thunder lock: enabled May 17 19:09:08.112150 dv-cinder-840678-10-iscsi devstack@keystone.service[80151]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 May 17 19:09:08.112150 dv-cinder-840678-10-iscsi devstack@keystone.service[80151]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] May 17 19:09:08.157667 dv-cinder-840678-10-iscsi devstack@keystone.service[80151]: Python main interpreter initialized at 0x560c22fa79c0 May 17 19:09:08.157667 dv-cinder-840678-10-iscsi devstack@keystone.service[80151]: python threads support enabled May 17 19:09:08.157667 dv-cinder-840678-10-iscsi devstack@keystone.service[80151]: your server socket listen backlog is limited to 100 connections May 17 19:09:08.157930 dv-cinder-840678-10-iscsi devstack@keystone.service[80151]: your mercy for graceful operations on workers is 90 seconds May 17 19:09:08.158004 dv-cinder-840678-10-iscsi devstack@keystone.service[80151]: mapped 403077 bytes (393 KB) for 2 cores May 17 19:09:08.158075 dv-cinder-840678-10-iscsi devstack@keystone.service[80151]: *** Operational MODE: preforking *** May 17 19:09:08.158364 dv-cinder-840678-10-iscsi devstack@keystone.service[80151]: *** uWSGI is running in multiple interpreter mode *** May 17 19:09:08.158364 dv-cinder-840678-10-iscsi devstack@keystone.service[80151]: spawned uWSGI master process (pid: 80151) May 17 19:09:08.158547 dv-cinder-840678-10-iscsi systemd[1]: Started Devstack devstack@keystone.service. May 17 19:09:08.159928 dv-cinder-840678-10-iscsi devstack@keystone.service[80151]: spawned uWSGI worker 1 (pid: 80160, cores: 1) May 17 19:09:08.159928 dv-cinder-840678-10-iscsi devstack@keystone.service[80151]: spawned uWSGI worker 2 (pid: 80161, cores: 1) May 17 19:09:08.159928 dv-cinder-840678-10-iscsi devstack@keystone.service[80151]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... May 17 19:09:13.800093 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 17 19:09:13.800093 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: from cryptography.utils import int_from_bytes May 17 19:09:13.801350 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 17 19:09:13.801350 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: from cryptography.utils import int_from_bytes May 17 19:09:13.801872 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 17 19:09:13.801872 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: from cryptography.utils import int_from_bytes May 17 19:09:13.802310 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 17 19:09:13.802310 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: from cryptography.utils import int_from_bytes May 17 19:09:13.850137 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 17 19:09:13.850330 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 17 19:09:13.850506 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] command line args: [] {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 17 19:09:13.850777 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 17 19:09:13.850955 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ================================================================================ {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 17 19:09:13.851040 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 17 19:09:13.851209 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] admin_token = **** {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 17 19:09:13.851295 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 17 19:09:13.851407 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] command line args: [] {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 17 19:09:13.851519 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] config_dir = [] {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 17 19:09:13.851650 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 17 19:09:13.851737 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 17 19:09:13.851820 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] config_source = [] {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 17 19:09:13.851897 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ================================================================================ {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 17 19:09:13.851999 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] debug = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 17 19:09:13.852112 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] admin_token = **** {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 17 19:09:13.852191 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 17 19:09:13.852505 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] config_dir = [] {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 17 19:09:13.852505 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 17 19:09:13.852704 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 17 19:09:13.852807 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] config_source = [] {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 17 19:09:13.852909 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 17 19:09:13.853011 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] debug = True {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 17 19:09:13.853128 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 17 19:09:13.853232 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 17 19:09:13.853359 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 17 19:09:13.853971 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 17 19:09:13.854076 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] list_limit = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 17 19:09:13.854192 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 17 19:09:13.854296 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] log_config_append = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 17 19:09:13.854398 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 17 19:09:13.854500 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 17 19:09:13.854630 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 17 19:09:13.854745 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] log_dir = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 17 19:09:13.854745 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] log_file = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 17 19:09:13.854938 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] list_limit = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 17 19:09:13.854938 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] log_config_append = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 17 19:09:13.855084 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 17 19:09:13.855161 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 17 19:09:13.855244 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 17 19:09:13.855315 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] log_dir = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 17 19:09:13.855315 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] log_file = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 17 19:09:13.855442 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 17 19:09:13.855545 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 17 19:09:13.855625 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 17 19:09:13.855712 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 17 19:09:13.855785 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 17 19:09:13.855911 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 17 19:09:13.855993 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 17 19:09:13.856130 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 17 19:09:13.856262 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 17 19:09:13.856388 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 17 19:09:13.856451 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(user_domain)s %(project_domain)s {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 17 19:09:13.856517 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 17 19:09:13.856581 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 17 19:09:13.856808 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 17 19:09:13.856921 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 17 19:09:13.856990 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(user_domain)s %(project_domain)s {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 17 19:09:13.857053 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 17 19:09:13.857240 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 17 19:09:13.857337 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 17 19:09:13.857494 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 17 19:09:13.857586 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 17 19:09:13.857668 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 17 19:09:13.857758 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 17 19:09:13.857954 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 17 19:09:13.858058 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 17 19:09:13.858127 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 17 19:09:13.858366 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 17 19:09:13.858465 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.58/identity {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 17 19:09:13.858654 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] publish_errors = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 17 19:09:13.858756 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 17 19:09:13.858928 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 17 19:09:13.859121 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 17 19:09:13.859222 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.58/identity {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 17 19:09:13.859285 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 17 19:09:13.859354 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] publish_errors = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 17 19:09:13.859564 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 17 19:09:13.859661 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 17 19:09:13.859819 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 17 19:09:13.859913 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 17 19:09:13.860065 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 17 19:09:13.860137 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] standard_threads = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 17 19:09:13.860202 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 17 19:09:13.860261 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 17 19:09:13.860466 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 17 19:09:13.860572 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 17 19:09:13.860572 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 17 19:09:13.860693 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] standard_threads = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 17 19:09:13.860756 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] use_journal = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 17 19:09:13.860814 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 17 19:09:13.860993 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] use_json = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 17 19:09:13.861121 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 17 19:09:13.861211 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] use_stderr = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 17 19:09:13.861281 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 17 19:09:13.861340 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] use_syslog = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 17 19:09:13.861401 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] use_journal = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 17 19:09:13.861464 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 17 19:09:13.861532 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] use_json = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 17 19:09:13.861696 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] use_stderr = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 17 19:09:13.861771 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.861855 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] use_syslog = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 17 19:09:13.861987 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.862068 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 17 19:09:13.862182 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.862326 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.862418 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.862564 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.862654 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.862820 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.862908 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.863058 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.863142 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.863305 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.863381 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.863565 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.863667 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.863814 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.863899 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.864047 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.864124 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.864293 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.864444 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.864541 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.864618 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.864779 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.864857 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.865009 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.865099 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.865253 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.865336 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.865486 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.865563 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.865730 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.865819 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.865975 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.866051 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.866208 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.866284 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.866447 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.866532 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.866699 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.866775 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.866942 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.867025 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.867175 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.867261 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.867427 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.867530 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.867725 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.867803 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.867975 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.868067 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.868206 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.868291 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.868437 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.868519 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.868665 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.868774 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.870218 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.870328 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.870408 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.870526 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.870613 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.870686 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.870806 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.870881 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.871002 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.871077 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.871199 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.871313 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.871397 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.871576 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.871655 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.871771 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.871858 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.871931 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.872052 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.872128 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.872250 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.872332 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.872406 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.872529 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.872610 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.872681 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.872757 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.872873 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.872996 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.873072 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.873198 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.873275 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.873346 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.873452 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.873537 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.873620 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.873697 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.873768 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.873846 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.873964 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.874079 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.874165 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.874237 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.874351 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.874425 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.874496 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.874612 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.874698 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.874770 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.874884 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.874997 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.875124 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.875241 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.875362 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.875477 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.875616 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.875730 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.875839 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.875959 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.876094 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.876180 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.876331 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.876433 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.876569 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.876647 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.876804 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.876883 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.877066 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.877147 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.877302 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.877391 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.877533 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.877618 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.877773 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.877850 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.878000 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.878075 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.878235 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.878321 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.878471 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.878550 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.878734 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.878844 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.878970 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.879095 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.879216 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.879337 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.879456 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.879590 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.879714 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.879826 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.879946 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.880079 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.880178 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.880321 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.880405 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.880560 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.880647 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.880806 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.880885 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.881044 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.881122 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.881284 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.881370 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.881522 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.881601 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.881752 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.881828 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.881984 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.882070 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.882216 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.882294 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.882473 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.882551 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.882710 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.882796 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.882945 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.883022 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.883177 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.883260 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.883418 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.883522 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.883675 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.883752 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.883911 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.883987 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.884140 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.884231 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.884370 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.884447 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.884594 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.884883 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.884883 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.885043 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.885118 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.885189 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.885301 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.885421 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.885538 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.885663 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.885787 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.885899 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.886025 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.886145 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.886285 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.886365 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.886545 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.886645 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.886787 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.886873 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.887019 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] auth.external = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.887099 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.887251 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.887336 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] auth.external = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.887547 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.887628 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.887788 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.887876 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.888019 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] auth.password = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.888099 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.888249 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] auth.token = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.888328 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] auth.password = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.888488 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.888566 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] auth.token = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.888723 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.888802 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.888957 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.889035 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.889186 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.889263 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.889418 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.889502 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.889664 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.889744 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.889896 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.889973 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.890122 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.890202 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] credential.caching = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.890344 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] credential.caching = True {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.890424 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.890590 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.890668 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.890835 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.890915 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.891072 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.891150 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.891303 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.891379 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.891566 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.891648 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.891802 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.891882 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.892078 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.892160 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.892318 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.892399 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.892575 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.892654 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.892825 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.892902 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.893094 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.893173 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.893347 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.893424 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.893588 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.893666 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.893836 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.893967 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.894092 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] federation.caching = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.894213 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.894336 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.894458 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] federation.caching = True {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.894593 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.894718 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.894847 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.894982 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.894982 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.895215 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.895215 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.895458 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.895604 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.895682 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.895755 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.895826 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.895910 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.895983 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.896152 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.896230 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.896400 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.896481 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.896631 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.896710 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.896872 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] identity.caching = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.896949 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.897103 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] identity.caching = True {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.897189 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.897347 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.897426 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.897581 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.897658 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.897812 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.897898 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.898044 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.898123 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.898276 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.898353 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.898504 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.898590 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.898738 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.898817 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.898982 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.899061 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.899213 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.899298 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.899443 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.899629 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.899706 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.899789 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.899910 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.899997 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.900150 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.900230 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.900421 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.900500 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.900664 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.900755 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.900910 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.900987 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.901155 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.901272 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.901414 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.901522 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.901653 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.901739 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.901883 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.901983 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.902113 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.902212 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.902343 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.902423 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.902569 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.902652 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.902801 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.902899 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.903038 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.903123 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.903274 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.903356 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.903525 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.903623 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.903759 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.903839 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.904003 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.904083 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.904240 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.904328 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.904470 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.904550 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.904713 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.904794 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.904944 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.905033 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.905172 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.905251 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.905400 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.905481 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.905624 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.905712 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.905856 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.905937 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.906089 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.906170 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.906325 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.906418 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.906556 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.906637 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.906800 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.906880 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.907046 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.907135 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.907279 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.907360 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.907523 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.907607 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.907759 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.907851 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.908005 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.908086 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.908239 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.908318 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.908471 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.908560 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.908705 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.908789 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.908934 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.909013 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.user = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.909163 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.user = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.909244 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.909400 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.909480 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.909643 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.909723 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.909871 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.909951 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.910113 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.910193 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.910354 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.910434 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.910588 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.910668 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.910817 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.910896 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.911042 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.911122 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.911273 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.911354 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.911522 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.911606 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.911756 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.911837 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.911981 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.912062 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.912224 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.912305 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.912467 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.912547 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.912700 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.912778 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.912935 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.913016 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.913164 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.913250 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.913391 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.913478 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.913632 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.913712 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.913864 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.913953 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.914096 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.914177 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.914327 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.914407 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.914557 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.914637 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.914797 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.914879 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.915031 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.915110 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.915260 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.915339 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.915523 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.915606 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.915755 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.915834 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.916530 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.916622 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.916765 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.916852 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.916999 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.917080 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.917231 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.917311 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.917475 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.917555 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.917710 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.917790 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.917933 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.918019 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.918166 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.918248 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] resource.caching = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.918401 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] resource.caching = True {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.918482 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.918648 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.918732 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.918883 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.918971 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.919112 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.919192 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.919369 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.919448 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.919634 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.919757 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.919868 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.919947 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.920108 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.920188 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.920340 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.920421 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.920574 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] role.caching = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.920653 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.920805 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] role.caching = True {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.920884 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] role.driver = sql {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.921040 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.921120 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] role.driver = sql {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.921266 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.921346 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.921511 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.921592 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.921755 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.921837 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.921998 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.922077 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.922241 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.922328 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.922483 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.922565 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.922724 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.922804 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.922964 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.923052 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.923193 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.923279 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.923424 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.923532 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.923693 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.923786 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.923938 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.924018 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.924179 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.924258 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.924422 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.924572 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.924810 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.924889 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.925045 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.925125 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.925296 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.925386 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.925542 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.925621 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.925787 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.925866 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.926021 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.926109 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.926256 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.926335 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.926506 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.926593 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.926748 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.926836 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.926982 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.927062 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.927210 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.927289 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.927439 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.927540 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.927711 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.927791 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.927961 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.928042 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.928207 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.928288 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.928447 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.928528 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.928678 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.928757 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.928907 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.928987 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.929133 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.929211 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] token.caching = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.929365 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] token.caching = True {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.929445 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.929599 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.929681 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.929835 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.929914 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.930066 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.930145 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.930316 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.930403 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.930558 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.930638 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.930788 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.930875 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.931027 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.931113 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.931267 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.931348 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.931561 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.932192 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.932334 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.932465 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.932592 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.932718 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.932850 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.932977 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.933111 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.933250 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.933377 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.933503 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.933628 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.933753 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.933753 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.934000 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.934000 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] database.connection = **** {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.934249 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.934383 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.934511 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] database.connection = **** {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.934645 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.934779 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.934906 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.935054 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.935183 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.935309 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.935435 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.935609 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.935736 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.935874 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.936000 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.936124 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.936248 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.936376 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.936500 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.936636 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.936776 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.936906 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.937031 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.937155 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.937279 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.937411 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.937411 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.937646 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.937769 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.937892 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.938016 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.938148 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.938279 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.938403 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.938585 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.938711 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.938836 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.938968 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.938968 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 17 19:09:13.939212 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 17 19:09:13.939212 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 17 19:09:13.967698 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80161) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 17 19:09:13.967932 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80161) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 17 19:09:13.968156 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80161) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 17 19:09:13.968918 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 17 19:09:13.969154 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 17 19:09:13.969379 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 17 19:09:13.978931 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 17 19:09:13.979077 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80161) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 17 19:09:14.007561 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80161) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 17 19:09:14.007771 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80161) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 17 19:09:14.008058 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 17 19:09:14.008287 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 17 19:09:14.060821 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80161) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 17 19:09:14.061255 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 17 19:09:14.111975 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80161) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 17 19:09:14.112167 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 17 19:09:14.112321 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80161) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 17 19:09:14.112456 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 17 19:09:14.177854 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80161) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 17 19:09:14.178034 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 17 19:09:14.178165 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80161) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 17 19:09:14.178306 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 17 19:09:14.178433 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80161) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 17 19:09:14.178558 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 17 19:09:14.178683 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80161) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 17 19:09:14.178820 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 17 19:09:14.178945 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80161) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 17 19:09:14.179078 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 17 19:09:14.179204 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80161) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 17 19:09:14.179327 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 17 19:09:14.202538 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80161) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 17 19:09:14.202764 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80161) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 17 19:09:14.202904 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 17 19:09:14.203032 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80161) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 17 19:09:14.203156 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 17 19:09:14.203280 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80161) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 17 19:09:14.203416 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 17 19:09:14.203568 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80161) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 17 19:09:14.203695 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 17 19:09:14.203824 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80161) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 17 19:09:14.203951 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 17 19:09:14.204078 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80161) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 17 19:09:14.204205 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 17 19:09:14.204205 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 17 19:09:14.215754 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80161) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 17 19:09:14.215989 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 17 19:09:14.228697 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80161) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 17 19:09:14.228926 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80161) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 17 19:09:14.229149 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80161) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 17 19:09:14.229433 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80161) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 17 19:09:14.229653 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80161) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 17 19:09:14.229863 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80161) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 17 19:09:14.230072 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80161) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 17 19:09:14.230612 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 17 19:09:14.230847 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 17 19:09:14.231072 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 17 19:09:14.231292 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 17 19:09:14.231533 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 17 19:09:14.231751 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 17 19:09:14.231963 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 17 19:09:14.247696 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.247883 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.247969 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 17 19:09:14.248091 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 17 19:09:14.248221 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.248422 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 17 19:09:14.248505 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.248623 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 17 19:09:14.248727 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.248890 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.248974 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.249146 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.249226 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.249390 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.249470 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.249640 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.261160 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.261431 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.261696 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.261976 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.262668 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.262939 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.263202 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.263477 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.270064 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80160) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 17 19:09:14.271624 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80161) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 17 19:09:14.273354 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80160) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 17 19:09:14.273633 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.273912 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.274300 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.274584 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.274846 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.274927 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80161) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 17 19:09:14.275115 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.275196 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.275411 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.275518 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.275705 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.275825 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.275978 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.276108 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.276245 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.276364 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.276624 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.276888 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.277151 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.277416 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.277677 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.302689 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.304268 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.304362 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80160) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 17 19:09:14.304583 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.305880 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80161) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 17 19:09:14.306160 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.310332 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80160) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 17 19:09:14.310608 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.310931 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.311958 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80161) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 17 19:09:14.312232 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.312512 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.318154 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80160) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 17 19:09:14.318436 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.319793 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80161) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 17 19:09:14.320070 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.323597 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80160) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 17 19:09:14.323878 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.324153 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.324431 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.324702 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.324974 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.325224 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80161) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 17 19:09:14.325307 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.325491 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.325573 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.325767 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.326045 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.326315 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.326588 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.326855 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.327119 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.344026 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.344298 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.345567 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.345839 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.349054 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80160) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 17 19:09:14.350561 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80161) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 17 19:09:14.352665 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.352957 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.354673 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.354959 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.358402 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80160) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 17 19:09:14.359899 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80161) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 17 19:09:14.361983 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80160) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 17 19:09:14.363418 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80161) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 17 19:09:14.365750 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.366042 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.366319 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.366596 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.366868 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.367111 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.367194 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.367398 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.367698 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.367974 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.368244 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.368553 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.386663 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80160) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 17 19:09:14.387030 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.387297 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.387591 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.387947 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80161) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 17 19:09:14.388315 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.388578 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.388845 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.395447 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.396682 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.397190 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.397461 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.398368 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.398636 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.400576 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80160) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 17 19:09:14.400849 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.401138 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.401418 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.401699 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.401793 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80161) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 17 19:09:14.402000 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.402273 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.402562 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.402833 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.415030 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80160) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 17 19:09:14.415320 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.415622 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.416439 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.416592 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80161) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 17 19:09:14.416723 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.416863 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.417005 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.417134 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.417280 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.417407 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.417680 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.417964 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.418240 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.435466 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80160) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 17 19:09:14.436870 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80161) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 17 19:09:14.438716 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80160) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 17 19:09:14.440110 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80161) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 17 19:09:14.442011 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.443359 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.443630 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.444971 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.445260 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80160) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 17 19:09:14.445542 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.445830 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.446597 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80161) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 17 19:09:14.446875 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.447701 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.453757 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.455034 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.455365 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80160) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 17 19:09:14.456658 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80161) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 17 19:09:14.458596 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.458882 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.459163 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.459435 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.459872 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.460157 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.460432 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.460702 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.468741 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80160) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 17 19:09:14.469071 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.469390 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.470958 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80161) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 17 19:09:14.471239 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.471534 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.477532 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80160) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 17 19:09:14.477807 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.478090 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.478867 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.479146 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.479434 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.479738 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.480005 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.480134 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80161) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 17 19:09:14.480284 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.480412 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.480539 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.480674 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.480814 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.480941 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.481067 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.481196 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.481330 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.481457 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.481584 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.481718 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.481923 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.482200 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.482469 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.482732 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.482991 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.483248 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.483527 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 17 19:09:14.520542 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x560c22fa79c0 pid: 80160 (default app) May 17 19:09:14.522193 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x560c22fa79c0 pid: 80161 (default app) May 17 19:09:14.523905 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e3c07b-17aa-4a33-9fad-f517acbd8bdf None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:14.524203 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e3c07b-17aa-4a33-9fad-f517acbd8bdf None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:14.524481 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e3c07b-17aa-4a33-9fad-f517acbd8bdf None None] PATH_INFO: `/v3` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:14.528698 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1/1] 10.222.0.58 () {50 vars in 791 bytes} [Tue May 17 19:09:14 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:09:23.436848 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4381761b-7f9a-4a4e-98a6-04ab0b1780f1 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:23.437336 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4381761b-7f9a-4a4e-98a6-04ab0b1780f1 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:23.437336 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4381761b-7f9a-4a4e-98a6-04ab0b1780f1 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:23.441645 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1/2] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:09:23 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 17 19:09:23.446854 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d855a137-ee25-41a4-987e-919a44a531dd None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:23.447215 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d855a137-ee25-41a4-987e-919a44a531dd None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:23.447401 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d855a137-ee25-41a4-987e-919a44a531dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:23.477294 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG oslo_db.sqlalchemy.engines [None req-d855a137-ee25-41a4-987e-919a44a531dd None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80160) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 17 19:09:23.659095 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG passlib.handlers.bcrypt [None req-d855a137-ee25-41a4-987e-919a44a531dd None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80160) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 17 19:09:23.659533 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG passlib.handlers.bcrypt [None req-d855a137-ee25-41a4-987e-919a44a531dd None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80160) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 17 19:09:23.714878 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-d855a137-ee25-41a4-987e-919a44a531dd None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:09:23.716485 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d855a137-ee25-41a4-987e-919a44a531dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:09:23.839986 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 2/3] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:09:23 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 394 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) May 17 19:09:23.997693 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4381761b-7f9a-4a4e-98a6-04ab0b1780f1 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:24.031240 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG oslo_db.sqlalchemy.engines [None req-4381761b-7f9a-4a4e-98a6-04ab0b1780f1 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80161) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 17 19:09:24.156847 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4381761b-7f9a-4a4e-98a6-04ab0b1780f1 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:24.161110 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de86447-70f3-4ec6-9d69-0ed6004a674b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:24.161767 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de86447-70f3-4ec6-9d69-0ed6004a674b admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:24.162049 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de86447-70f3-4ec6-9d69-0ed6004a674b admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:24.162320 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de86447-70f3-4ec6-9d69-0ed6004a674b admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:24.170651 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de86447-70f3-4ec6-9d69-0ed6004a674b admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:24.221938 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: WARNING keystone.common.rbac_enforcer.enforcer [None req-6de86447-70f3-4ec6-9d69-0ed6004a674b admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 17 19:09:24.222534 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de86447-70f3-4ec6-9d69-0ed6004a674b admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:24.231768 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 2/4] 10.222.0.58 () {58 vars in 1324 bytes} [Tue May 17 19:09:23 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 17 19:09:24.237618 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d855a137-ee25-41a4-987e-919a44a531dd None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:24.251537 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d855a137-ee25-41a4-987e-919a44a531dd None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:24.255843 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd68a2fd-1852-40b7-a90c-9cb7140bd0a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:24.256920 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd68a2fd-1852-40b7-a90c-9cb7140bd0a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:24.257201 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd68a2fd-1852-40b7-a90c-9cb7140bd0a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:24.257476 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd68a2fd-1852-40b7-a90c-9cb7140bd0a9 admin admin] PATH_INFO: `/v3/projects/07bd98f7df3a477a8bf75754083485f0` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:24.258951 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd68a2fd-1852-40b7-a90c-9cb7140bd0a9 admin admin] RBAC: Authorizing `identity:get_project(project_id=07bd98f7df3a477a8bf75754083485f0)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:24.308507 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: WARNING keystone.common.rbac_enforcer.enforcer [None req-dd68a2fd-1852-40b7-a90c-9cb7140bd0a9 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 17 19:09:24.308848 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd68a2fd-1852-40b7-a90c-9cb7140bd0a9 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:24.313371 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 3/5] 10.222.0.58 () {56 vars in 1207 bytes} [Tue May 17 19:09:24 2022] GET /identity/v3/projects/07bd98f7df3a477a8bf75754083485f0 => generated 344 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:09:26.460709 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5206315a-e4a4-4dc4-9342-bed8edb20c4f None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:26.461207 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5206315a-e4a4-4dc4-9342-bed8edb20c4f None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:26.461352 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5206315a-e4a4-4dc4-9342-bed8edb20c4f None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:26.465480 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 3/6] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:09:26 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 17 19:09:26.471012 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a2b008-5091-47ba-82f7-81649e67e9f9 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:26.471376 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a2b008-5091-47ba-82f7-81649e67e9f9 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:26.471591 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a2b008-5091-47ba-82f7-81649e67e9f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:26.491334 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-28a2b008-5091-47ba-82f7-81649e67e9f9 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:09:26.492864 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-28a2b008-5091-47ba-82f7-81649e67e9f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:09:26.501029 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 4/7] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:09:26 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 17 19:09:26.523017 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5206315a-e4a4-4dc4-9342-bed8edb20c4f None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:26.534929 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5206315a-e4a4-4dc4-9342-bed8edb20c4f None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:26.539092 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5df989d-1060-4966-af37-6e391d9f50d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:26.539737 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5df989d-1060-4966-af37-6e391d9f50d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:26.540019 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5df989d-1060-4966-af37-6e391d9f50d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:26.540294 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5df989d-1060-4966-af37-6e391d9f50d3 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:26.547767 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5df989d-1060-4966-af37-6e391d9f50d3 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:26.549529 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5df989d-1060-4966-af37-6e391d9f50d3 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:26.558610 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 4/8] 10.222.0.58 () {58 vars in 1324 bytes} [Tue May 17 19:09:26 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 17 19:09:26.563791 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a2b008-5091-47ba-82f7-81649e67e9f9 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:26.572963 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a2b008-5091-47ba-82f7-81649e67e9f9 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:26.577395 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecc7dd5-970a-4ec2-ae46-6c3fe1f8534f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:26.578253 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecc7dd5-970a-4ec2-ae46-6c3fe1f8534f admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:26.578537 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecc7dd5-970a-4ec2-ae46-6c3fe1f8534f admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:26.578812 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecc7dd5-970a-4ec2-ae46-6c3fe1f8534f admin admin] PATH_INFO: `/v3/users/ac2441e330cd41aaa04d125461331f3b` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:26.580253 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecc7dd5-970a-4ec2-ae46-6c3fe1f8534f admin admin] RBAC: Authorizing `identity:get_user(user_id=ac2441e330cd41aaa04d125461331f3b)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:26.582069 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecc7dd5-970a-4ec2-ae46-6c3fe1f8534f admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:26.583653 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 5/9] 10.222.0.58 () {56 vars in 1198 bytes} [Tue May 17 19:09:26 2022] GET /identity/v3/users/ac2441e330cd41aaa04d125461331f3b => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:09:28.827233 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef923ce1-8bb7-4495-a0d3-d6663d97090c None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:28.827791 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef923ce1-8bb7-4495-a0d3-d6663d97090c None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:28.827791 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef923ce1-8bb7-4495-a0d3-d6663d97090c None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:28.829133 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 5/10] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:09:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:09:28.834392 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2990cf63-7622-402e-8161-6f0678a053a4 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:28.834793 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2990cf63-7622-402e-8161-6f0678a053a4 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:28.834950 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2990cf63-7622-402e-8161-6f0678a053a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:28.837900 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7e388b-9f4c-4dba-a02f-580a28c6fdd8 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:28.838182 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7e388b-9f4c-4dba-a02f-580a28c6fdd8 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:28.838457 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7e388b-9f4c-4dba-a02f-580a28c6fdd8 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:28.839681 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 6/11] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:09:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 17 19:09:28.844705 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e35203d-30b9-4c8a-b06e-58c2814d351d None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:28.844986 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e35203d-30b9-4c8a-b06e-58c2814d351d None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:28.845259 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e35203d-30b9-4c8a-b06e-58c2814d351d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:28.886779 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG passlib.handlers.bcrypt [None req-0e35203d-30b9-4c8a-b06e-58c2814d351d None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80161) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 17 19:09:28.887192 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG passlib.handlers.bcrypt [None req-0e35203d-30b9-4c8a-b06e-58c2814d351d None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80161) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 17 19:09:28.934776 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-0e35203d-30b9-4c8a-b06e-58c2814d351d None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:09:28.936427 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-0e35203d-30b9-4c8a-b06e-58c2814d351d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:09:28.959966 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-2990cf63-7622-402e-8161-6f0678a053a4 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:09:28.961158 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-2990cf63-7622-402e-8161-6f0678a053a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:09:29.037973 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 6/12] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:09:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 17 19:09:29.046695 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 7/13] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:09:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 17 19:09:29.061224 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2990cf63-7622-402e-8161-6f0678a053a4 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:29.067776 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e35203d-30b9-4c8a-b06e-58c2814d351d None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:29.156192 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2990cf63-7622-402e-8161-6f0678a053a4 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:29.163094 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e35203d-30b9-4c8a-b06e-58c2814d351d None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:29.186910 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7861d7-08dd-4443-9943-65248b5833c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:29.187717 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7861d7-08dd-4443-9943-65248b5833c8 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:29.187996 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7861d7-08dd-4443-9943-65248b5833c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:29.188263 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7861d7-08dd-4443-9943-65248b5833c8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:29.194355 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433b2bf7-c315-4eb0-9d62-f065c6e20d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:29.195232 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433b2bf7-c315-4eb0-9d62-f065c6e20d35 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:29.195542 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433b2bf7-c315-4eb0-9d62-f065c6e20d35 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:29.195821 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433b2bf7-c315-4eb0-9d62-f065c6e20d35 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:29.199597 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7861d7-08dd-4443-9943-65248b5833c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 17 19:09:29.210537 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-fd7861d7-08dd-4443-9943-65248b5833c8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 17 19:09:29.210537 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:09:29.210537 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:09:29.210537 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:09:29.210537 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:09:29.210537 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:09:29.210537 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:09:29.210537 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:09:29.210537 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:09:29.210537 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:09:29.210537 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:09:29.210537 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:09:29.210537 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 17 19:09:29.210537 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) May 17 19:09:29.210537 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 17 19:09:29.210537 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 17 19:09:29.211370 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 17 19:09:29.211370 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 17 19:09:29.211370 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 17 19:09:29.211370 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 17 19:09:29.211370 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:09:29.211370 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:09:29.211370 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:09:29.211370 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:09:29.211370 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:09:29.211370 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:09:29.211370 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:09:29.211370 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:09:29.211370 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:09:29.211370 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:09:29.211370 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:09:29.211370 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:09:29.213043 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:09:29.213043 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:09:29.213043 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:09:29.213043 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:09:29.213043 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 17 19:09:29.213043 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 17 19:09:29.213043 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 17 19:09:29.213043 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 17 19:09:29.213043 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 17 19:09:29.213043 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 17 19:09:29.213043 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 17 19:09:29.213043 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:09:29.213043 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 7/14] 10.222.0.58 () {56 vars in 1096 bytes} [Tue May 17 19:09:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:09:29.217945 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7861d7-08dd-4443-9943-65248b5833c8 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:29.260189 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433b2bf7-c315-4eb0-9d62-f065c6e20d35 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:29.261697 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433b2bf7-c315-4eb0-9d62-f065c6e20d35 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:29.310854 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7861d7-08dd-4443-9943-65248b5833c8 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:29.343393 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f9637d-be36-4ae1-bfbe-e547253cc922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:29.344065 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f9637d-be36-4ae1-bfbe-e547253cc922 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:29.344344 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f9637d-be36-4ae1-bfbe-e547253cc922 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:29.344614 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f9637d-be36-4ae1-bfbe-e547253cc922 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:29.345354 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f9637d-be36-4ae1-bfbe-e547253cc922 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:09:29.345656 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f9637d-be36-4ae1-bfbe-e547253cc922 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:29.347296 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f9637d-be36-4ae1-bfbe-e547253cc922 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:29.352134 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 8/15] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 17 19:09:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 17 19:09:29.357279 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433b2bf7-c315-4eb0-9d62-f065c6e20d35 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:29.363136 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 8/16] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 17 19:09:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:09:29.363987 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f9637d-be36-4ae1-bfbe-e547253cc922 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:29.450924 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433b2bf7-c315-4eb0-9d62-f065c6e20d35 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:29.460618 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f9637d-be36-4ae1-bfbe-e547253cc922 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:29.483610 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab5cb4e-9e54-48dc-a771-d2b0ab78488c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:29.484397 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab5cb4e-9e54-48dc-a771-d2b0ab78488c None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:29.484675 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab5cb4e-9e54-48dc-a771-d2b0ab78488c None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:29.484950 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab5cb4e-9e54-48dc-a771-d2b0ab78488c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:29.494159 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3394b49f-f7fa-4e9d-ba7a-004585d3e7d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:29.495012 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3394b49f-f7fa-4e9d-ba7a-004585d3e7d5 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:29.495287 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3394b49f-f7fa-4e9d-ba7a-004585d3e7d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:29.495579 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3394b49f-f7fa-4e9d-ba7a-004585d3e7d5 None admin] PATH_INFO: `/v3/users/ac2441e330cd41aaa04d125461331f3b` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:29.496765 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3394b49f-f7fa-4e9d-ba7a-004585d3e7d5 None admin] RBAC: Authorizing `identity:get_user(user_id=ac2441e330cd41aaa04d125461331f3b)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:29.498213 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3394b49f-f7fa-4e9d-ba7a-004585d3e7d5 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:29.499837 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 9/17] 10.222.0.58 () {56 vars in 1177 bytes} [Tue May 17 19:09:29 2022] GET /identity/v3/users/ac2441e330cd41aaa04d125461331f3b => generated 247 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:09:29.503083 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab5cb4e-9e54-48dc-a771-d2b0ab78488c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:29.504574 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab5cb4e-9e54-48dc-a771-d2b0ab78488c None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:29.505207 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3394b49f-f7fa-4e9d-ba7a-004585d3e7d5 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:29.525471 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-1ab5cb4e-9e54-48dc-a771-d2b0ab78488c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:09:29.525471 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:09:29.525471 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:09:29.525471 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:09:29.525471 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:09:29.525471 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:09:29.525471 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:09:29.525471 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:09:29.525471 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:09:29.525471 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:09:29.525471 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:09:29.525471 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:09:29.525471 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 17 19:09:29.525471 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 17 19:09:29.525471 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 17 19:09:29.526482 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 17 19:09:29.526482 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:09:29.526482 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:09:29.526482 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:09:29.526482 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:09:29.526482 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:09:29.526482 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:09:29.526482 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:09:29.526482 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:09:29.526482 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:09:29.526482 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:09:29.526482 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:09:29.526482 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:09:29.526482 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:09:29.526482 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:09:29.526482 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:09:29.526482 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:09:29.527268 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 17 19:09:29.527268 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 17 19:09:29.527268 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:09:29.527268 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:09:29.527268 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 9/18] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:09:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 17 19:09:29.532290 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab5cb4e-9e54-48dc-a771-d2b0ab78488c None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:29.597695 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3394b49f-f7fa-4e9d-ba7a-004585d3e7d5 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:29.624996 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab5cb4e-9e54-48dc-a771-d2b0ab78488c None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:29.629234 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7241169-bf8d-4dbf-b47f-cea1aa8514d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:29.630018 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7241169-bf8d-4dbf-b47f-cea1aa8514d4 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:29.630295 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7241169-bf8d-4dbf-b47f-cea1aa8514d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:29.630563 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7241169-bf8d-4dbf-b47f-cea1aa8514d4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:29.631762 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7241169-bf8d-4dbf-b47f-cea1aa8514d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:29.633197 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7241169-bf8d-4dbf-b47f-cea1aa8514d4 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:29.635340 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 10/19] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:09:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:09:29.640954 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7241169-bf8d-4dbf-b47f-cea1aa8514d4 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:29.657193 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f375613b-1523-4717-95db-d884d2d9f729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:29.657928 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f375613b-1523-4717-95db-d884d2d9f729 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:29.658278 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f375613b-1523-4717-95db-d884d2d9f729 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:29.658621 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f375613b-1523-4717-95db-d884d2d9f729 None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:29.659413 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f375613b-1523-4717-95db-d884d2d9f729 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:09:29.659818 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f375613b-1523-4717-95db-d884d2d9f729 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:29.661220 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f375613b-1523-4717-95db-d884d2d9f729 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:29.677571 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 10/20] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 17 19:09:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:09:29.738089 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7241169-bf8d-4dbf-b47f-cea1aa8514d4 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:29.773367 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468b296d-06f9-4966-baa6-dd2676e3dd98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:29.774032 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468b296d-06f9-4966-baa6-dd2676e3dd98 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:29.774312 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468b296d-06f9-4966-baa6-dd2676e3dd98 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:29.774628 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468b296d-06f9-4966-baa6-dd2676e3dd98 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:29.775474 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468b296d-06f9-4966-baa6-dd2676e3dd98 None admin] RBAC: Adding query filter params (role.id=60af5088793e4379b88821a86552a344, scope.domain.id=default, user.id=ac2441e330cd41aaa04d125461331f3b) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:09:29.775808 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468b296d-06f9-4966-baa6-dd2676e3dd98 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:29.777264 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468b296d-06f9-4966-baa6-dd2676e3dd98 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:29.784646 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 11/21] 10.222.0.58 () {56 vars in 1322 bytes} [Tue May 17 19:09:29 2022] GET /identity/v3/role_assignments?user.id=ac2441e330cd41aaa04d125461331f3b&scope.domain.id=default&role.id=60af5088793e4379b88821a86552a344 => generated 233 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:09:32.324984 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001e4253-30c1-4634-b80e-31f42bb8604b None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:32.325460 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001e4253-30c1-4634-b80e-31f42bb8604b None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:32.325460 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001e4253-30c1-4634-b80e-31f42bb8604b None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:32.327122 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 11/22] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:09:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:09:32.332400 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25e8c0b-875d-49fd-843f-55dea1143610 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:32.332795 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25e8c0b-875d-49fd-843f-55dea1143610 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:32.332961 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25e8c0b-875d-49fd-843f-55dea1143610 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:32.353322 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-a25e8c0b-875d-49fd-843f-55dea1143610 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:09:32.354519 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a25e8c0b-875d-49fd-843f-55dea1143610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:09:32.442908 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 12/23] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:09:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 17 19:09:32.486411 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001e4253-30c1-4634-b80e-31f42bb8604b None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:32.609212 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001e4253-30c1-4634-b80e-31f42bb8604b None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:32.653204 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d459e2d-a57f-4c9d-b6c4-0ed79cbfc4ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:32.653879 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d459e2d-a57f-4c9d-b6c4-0ed79cbfc4ab None admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:32.654161 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d459e2d-a57f-4c9d-b6c4-0ed79cbfc4ab None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:32.654433 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d459e2d-a57f-4c9d-b6c4-0ed79cbfc4ab None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:32.655192 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d459e2d-a57f-4c9d-b6c4-0ed79cbfc4ab None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:32.656625 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d459e2d-a57f-4c9d-b6c4-0ed79cbfc4ab None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:32.674408 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ac5995-21f3-4e46-9c87-413352e5bcce None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:32.674756 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ac5995-21f3-4e46-9c87-413352e5bcce None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:32.675090 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ac5995-21f3-4e46-9c87-413352e5bcce None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:32.676841 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 13/24] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:09:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 17 19:09:32.682442 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2325dcfd-5fc6-4f40-a706-7d79876ffc03 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:32.682727 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2325dcfd-5fc6-4f40-a706-7d79876ffc03 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:32.683056 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2325dcfd-5fc6-4f40-a706-7d79876ffc03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:32.708809 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-2325dcfd-5fc6-4f40-a706-7d79876ffc03 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:09:32.710042 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-2325dcfd-5fc6-4f40-a706-7d79876ffc03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:09:32.817273 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 14/25] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:09:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 17 19:09:32.854125 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2325dcfd-5fc6-4f40-a706-7d79876ffc03 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:32.984910 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2325dcfd-5fc6-4f40-a706-7d79876ffc03 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:33.010801 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 12/26] 10.222.0.58 () {60 vars in 1131 bytes} [Tue May 17 19:09:32 2022] POST /identity/v3/roles => generated 219 bytes in 525 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:09:33.012485 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3abfdc-a3ad-43b7-bf10-99dd0a8489df None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:33.012773 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3abfdc-a3ad-43b7-bf10-99dd0a8489df None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:33.013047 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3abfdc-a3ad-43b7-bf10-99dd0a8489df None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:33.014598 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 13/27] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:09:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:09:33.016264 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759eb292-fb3c-44a5-b86b-55c3265023a8 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:33.016566 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759eb292-fb3c-44a5-b86b-55c3265023a8 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:33.016847 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759eb292-fb3c-44a5-b86b-55c3265023a8 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:33.018668 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 14/28] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:09:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:09:33.019864 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f762897-779e-40cd-b795-7f79a7f8df87 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:33.020205 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f762897-779e-40cd-b795-7f79a7f8df87 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:33.020482 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f762897-779e-40cd-b795-7f79a7f8df87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:33.028994 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fd1db8-a143-460b-a9c2-a13aefcb718f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:33.029691 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fd1db8-a143-460b-a9c2-a13aefcb718f None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:33.032599 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fd1db8-a143-460b-a9c2-a13aefcb718f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:33.032908 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fd1db8-a143-460b-a9c2-a13aefcb718f None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:33.033717 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fd1db8-a143-460b-a9c2-a13aefcb718f None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:33.035161 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fd1db8-a143-460b-a9c2-a13aefcb718f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:33.053597 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-7f762897-779e-40cd-b795-7f79a7f8df87 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:09:33.055437 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-7f762897-779e-40cd-b795-7f79a7f8df87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:09:33.166974 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 15/29] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:09:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 17 19:09:33.168445 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2c5f42-298a-463c-a0ee-8c82e049a20a None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:33.168736 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2c5f42-298a-463c-a0ee-8c82e049a20a None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:33.169014 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2c5f42-298a-463c-a0ee-8c82e049a20a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:33.204750 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-5a2c5f42-298a-463c-a0ee-8c82e049a20a None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:09:33.206366 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-5a2c5f42-298a-463c-a0ee-8c82e049a20a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:09:33.267722 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 15/30] 10.222.0.58 () {60 vars in 1131 bytes} [Tue May 17 19:09:32 2022] POST /identity/v3/roles => generated 225 bytes in 414 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:09:33.268846 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fd1db8-a143-460b-a9c2-a13aefcb718f None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:33.301022 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 16/31] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:09:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 17 19:09:33.334712 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2c5f42-298a-463c-a0ee-8c82e049a20a None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:33.394825 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fd1db8-a143-460b-a9c2-a13aefcb718f None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:33.447696 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2871a76d-9212-4509-ac57-e8e1ef11a003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:33.448941 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2871a76d-9212-4509-ac57-e8e1ef11a003 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:33.449290 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2871a76d-9212-4509-ac57-e8e1ef11a003 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:33.449575 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2871a76d-9212-4509-ac57-e8e1ef11a003 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:33.463010 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2871a76d-9212-4509-ac57-e8e1ef11a003 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:33.464604 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2871a76d-9212-4509-ac57-e8e1ef11a003 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:33.469447 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2c5f42-298a-463c-a0ee-8c82e049a20a None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:33.485608 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-2871a76d-9212-4509-ac57-e8e1ef11a003 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:09:33.485608 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:09:33.485608 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:09:33.485608 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:09:33.485608 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:09:33.485608 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:09:33.485608 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:09:33.485608 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:09:33.485608 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:09:33.485608 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:09:33.485608 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:09:33.485608 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:09:33.485608 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 17 19:09:33.485608 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 17 19:09:33.485608 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 17 19:09:33.486472 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 17 19:09:33.486472 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:09:33.486472 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:09:33.486472 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:09:33.486472 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:09:33.486472 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:09:33.486472 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:09:33.486472 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:09:33.486472 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:09:33.486472 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:09:33.486472 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:09:33.486472 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:09:33.486472 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:09:33.486472 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:09:33.486472 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:09:33.486472 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:09:33.486472 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:09:33.487757 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 17 19:09:33.487757 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 17 19:09:33.487757 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:09:33.487757 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:09:33.487757 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 16/32] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:09:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:09:33.490590 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa929e0a-c897-482b-8110-467a9ddce40d None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:33.490590 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa929e0a-c897-482b-8110-467a9ddce40d None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:33.490590 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa929e0a-c897-482b-8110-467a9ddce40d None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:33.490808 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 17/33] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:09:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:09:33.496989 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28c201b-0d12-4dc4-93e6-a2974a665cad None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:33.497275 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28c201b-0d12-4dc4-93e6-a2974a665cad None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:33.497552 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28c201b-0d12-4dc4-93e6-a2974a665cad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:33.516137 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-e28c201b-0d12-4dc4-93e6-a2974a665cad None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:09:33.516353 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d6ddd9-91ab-441d-a568-33db4cb8514c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:33.517165 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d6ddd9-91ab-441d-a568-33db4cb8514c None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:33.517472 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d6ddd9-91ab-441d-a568-33db4cb8514c None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:33.517719 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d6ddd9-91ab-441d-a568-33db4cb8514c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:33.518135 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-e28c201b-0d12-4dc4-93e6-a2974a665cad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:09:33.533454 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d6ddd9-91ab-441d-a568-33db4cb8514c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 17 19:09:33.543837 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-01d6ddd9-91ab-441d-a568-33db4cb8514c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 17 19:09:33.543837 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:09:33.543837 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:09:33.543837 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:09:33.543837 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:09:33.543837 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:09:33.543837 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:09:33.543837 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:09:33.543837 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:09:33.543837 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:09:33.543837 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:09:33.543837 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:09:33.543837 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 17 19:09:33.543837 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) May 17 19:09:33.543837 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 17 19:09:33.543837 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 17 19:09:33.545976 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 17 19:09:33.545976 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 17 19:09:33.545976 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 17 19:09:33.545976 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 17 19:09:33.545976 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:09:33.545976 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:09:33.545976 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:09:33.545976 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:09:33.545976 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:09:33.545976 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:09:33.545976 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:09:33.545976 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:09:33.545976 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:09:33.545976 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:09:33.545976 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:09:33.545976 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:09:33.547885 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:09:33.547885 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:09:33.547885 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:09:33.547885 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:09:33.547885 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 17 19:09:33.547885 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 17 19:09:33.547885 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 17 19:09:33.547885 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 17 19:09:33.547885 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 17 19:09:33.547885 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 17 19:09:33.547885 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 17 19:09:33.547885 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:09:33.547885 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 17/34] 10.222.0.58 () {56 vars in 1096 bytes} [Tue May 17 19:09:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 17 19:09:33.547885 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d6ddd9-91ab-441d-a568-33db4cb8514c None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:33.600981 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 18/35] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:09:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 17 19:09:33.601858 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28c201b-0d12-4dc4-93e6-a2974a665cad None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:33.668671 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d6ddd9-91ab-441d-a568-33db4cb8514c None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:33.701628 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2f418e-88c5-4615-a3ad-4f0664dc7a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:33.702296 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2f418e-88c5-4615-a3ad-4f0664dc7a4b None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:33.702592 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2f418e-88c5-4615-a3ad-4f0664dc7a4b None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:33.702865 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2f418e-88c5-4615-a3ad-4f0664dc7a4b None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:33.703622 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2f418e-88c5-4615-a3ad-4f0664dc7a4b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:09:33.703933 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2f418e-88c5-4615-a3ad-4f0664dc7a4b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:33.705460 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2f418e-88c5-4615-a3ad-4f0664dc7a4b None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:33.713186 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28c201b-0d12-4dc4-93e6-a2974a665cad None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:33.717526 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 18/36] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 17 19:09:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:09:33.718324 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2f418e-88c5-4615-a3ad-4f0664dc7a4b None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:33.749566 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7808a89e-75c3-4ec8-8ad1-74e830a32717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:33.750240 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7808a89e-75c3-4ec8-8ad1-74e830a32717 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:33.750522 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7808a89e-75c3-4ec8-8ad1-74e830a32717 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:33.750805 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7808a89e-75c3-4ec8-8ad1-74e830a32717 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:33.751600 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7808a89e-75c3-4ec8-8ad1-74e830a32717 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:09:33.751914 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7808a89e-75c3-4ec8-8ad1-74e830a32717 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:33.753313 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7808a89e-75c3-4ec8-8ad1-74e830a32717 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:33.767274 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 19/37] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 17 19:09:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:09:33.767274 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7808a89e-75c3-4ec8-8ad1-74e830a32717 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:33.822922 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2f418e-88c5-4615-a3ad-4f0664dc7a4b None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:33.861202 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b9a859-359e-4bb3-a039-5837d5613232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:33.861883 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b9a859-359e-4bb3-a039-5837d5613232 None admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:33.862165 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b9a859-359e-4bb3-a039-5837d5613232 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:33.862437 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b9a859-359e-4bb3-a039-5837d5613232 None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:33.863191 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b9a859-359e-4bb3-a039-5837d5613232 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:33.864607 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b9a859-359e-4bb3-a039-5837d5613232 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:33.867586 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7808a89e-75c3-4ec8-8ad1-74e830a32717 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:33.900246 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743d04a8-27af-4903-b227-227617105475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:33.900899 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743d04a8-27af-4903-b227-227617105475 None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:33.901182 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743d04a8-27af-4903-b227-227617105475 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:33.901455 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743d04a8-27af-4903-b227-227617105475 None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:33.902146 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743d04a8-27af-4903-b227-227617105475 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:33.903620 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743d04a8-27af-4903-b227-227617105475 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:33.949157 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 19/38] 10.222.0.58 () {60 vars in 1131 bytes} [Tue May 17 19:09:33 2022] POST /identity/v3/roles => generated 223 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:09:33.950038 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b9a859-359e-4bb3-a039-5837d5613232 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:34.007632 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 20/39] 10.222.0.58 () {60 vars in 1141 bytes} [Tue May 17 19:09:33 2022] POST /identity/v3/projects => generated 301 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:09:34.083984 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b9a859-359e-4bb3-a039-5837d5613232 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:34.124304 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6e4207-a32e-4df2-965b-968852bf2270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:34.125203 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6e4207-a32e-4df2-965b-968852bf2270 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:34.125490 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6e4207-a32e-4df2-965b-968852bf2270 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:34.125766 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6e4207-a32e-4df2-965b-968852bf2270 None admin] PATH_INFO: `/v3/users/ac2441e330cd41aaa04d125461331f3b` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:34.127545 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6e4207-a32e-4df2-965b-968852bf2270 None admin] RBAC: Authorizing `identity:get_user(user_id=ac2441e330cd41aaa04d125461331f3b)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:34.129042 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6e4207-a32e-4df2-965b-968852bf2270 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:34.131212 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 20/40] 10.222.0.58 () {56 vars in 1177 bytes} [Tue May 17 19:09:33 2022] GET /identity/v3/users/ac2441e330cd41aaa04d125461331f3b => generated 247 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:09:34.140676 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743d04a8-27af-4903-b227-227617105475 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:34.151830 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc473c13-9309-4e4c-aae9-72a3d4bbbad3 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:34.152123 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc473c13-9309-4e4c-aae9-72a3d4bbbad3 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:34.152392 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc473c13-9309-4e4c-aae9-72a3d4bbbad3 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:34.153912 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 21/41] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:09:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 17 19:09:34.160501 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a57b3ea-ffca-485d-b762-04d34d6a1244 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:34.160822 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a57b3ea-ffca-485d-b762-04d34d6a1244 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:34.161130 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a57b3ea-ffca-485d-b762-04d34d6a1244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:34.186622 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-0a57b3ea-ffca-485d-b762-04d34d6a1244 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:09:34.188034 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-0a57b3ea-ffca-485d-b762-04d34d6a1244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:09:34.264303 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743d04a8-27af-4903-b227-227617105475 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:34.270505 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 22/42] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:09:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 17 19:09:34.293979 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a57b3ea-ffca-485d-b762-04d34d6a1244 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:34.298917 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ceda4d-fd74-4461-aa7d-4bcae05cf626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:34.299766 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ceda4d-fd74-4461-aa7d-4bcae05cf626 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:34.300055 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ceda4d-fd74-4461-aa7d-4bcae05cf626 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:34.300336 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ceda4d-fd74-4461-aa7d-4bcae05cf626 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:34.301506 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ceda4d-fd74-4461-aa7d-4bcae05cf626 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:34.302968 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ceda4d-fd74-4461-aa7d-4bcae05cf626 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:34.304753 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 21/43] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:09:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:09:34.310476 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ceda4d-fd74-4461-aa7d-4bcae05cf626 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:34.385690 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a57b3ea-ffca-485d-b762-04d34d6a1244 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:34.403837 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ceda4d-fd74-4461-aa7d-4bcae05cf626 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:34.417786 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babb971c-e19d-4a93-9436-a591ce9658df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:34.418563 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babb971c-e19d-4a93-9436-a591ce9658df None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:34.418839 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babb971c-e19d-4a93-9436-a591ce9658df None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:34.419105 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babb971c-e19d-4a93-9436-a591ce9658df None admin] PATH_INFO: `/v3/domains/default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:34.420288 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babb971c-e19d-4a93-9436-a591ce9658df None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:34.421725 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babb971c-e19d-4a93-9436-a591ce9658df None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:34.423316 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 23/44] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:09:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:09:34.428601 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babb971c-e19d-4a93-9436-a591ce9658df None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:34.436979 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dff40cf-4608-485c-ba05-039b9cc20dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:34.437846 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dff40cf-4608-485c-ba05-039b9cc20dc3 None admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:34.438125 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dff40cf-4608-485c-ba05-039b9cc20dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:34.438406 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dff40cf-4608-485c-ba05-039b9cc20dc3 None admin] PATH_INFO: `/v3/domains/default/users/ac2441e330cd41aaa04d125461331f3b/roles/60af5088793e4379b88821a86552a344` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:34.440458 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dff40cf-4608-485c-ba05-039b9cc20dc3 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=ac2441e330cd41aaa04d125461331f3b, role_id=60af5088793e4379b88821a86552a344)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:34.441959 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dff40cf-4608-485c-ba05-039b9cc20dc3 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:34.513665 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 22/45] 10.222.0.58 () {58 vars in 1361 bytes} [Tue May 17 19:09:34 2022] PUT /identity/v3/domains/default/users/ac2441e330cd41aaa04d125461331f3b/roles/60af5088793e4379b88821a86552a344 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 17 19:09:34.522555 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babb971c-e19d-4a93-9436-a591ce9658df None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:34.559193 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d230ec5a-7b0d-4bf6-86ee-b5c7429ffa0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:34.559896 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d230ec5a-7b0d-4bf6-86ee-b5c7429ffa0f None admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:34.560177 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d230ec5a-7b0d-4bf6-86ee-b5c7429ffa0f None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:34.560448 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d230ec5a-7b0d-4bf6-86ee-b5c7429ffa0f None admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:34.561161 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d230ec5a-7b0d-4bf6-86ee-b5c7429ffa0f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:34.562619 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d230ec5a-7b0d-4bf6-86ee-b5c7429ffa0f None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:34.694322 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 24/46] 10.222.0.58 () {60 vars in 1141 bytes} [Tue May 17 19:09:34 2022] POST /identity/v3/projects => generated 312 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 17 19:09:36.713460 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b423e336-1524-4c40-a06e-41641bc6dd26 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:36.713996 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b423e336-1524-4c40-a06e-41641bc6dd26 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:36.713996 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b423e336-1524-4c40-a06e-41641bc6dd26 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:36.715421 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 23/47] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:09:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:09:36.721303 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af8b64d-7d1c-4570-9647-6e43f1cf4e84 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:36.721797 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af8b64d-7d1c-4570-9647-6e43f1cf4e84 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:36.721877 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af8b64d-7d1c-4570-9647-6e43f1cf4e84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:36.739783 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-7af8b64d-7d1c-4570-9647-6e43f1cf4e84 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:09:36.740958 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-7af8b64d-7d1c-4570-9647-6e43f1cf4e84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:09:36.803476 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 25/48] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:09:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 17 19:09:36.826985 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b423e336-1524-4c40-a06e-41641bc6dd26 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:36.879035 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8069bfca-9f4e-49d6-9ea8-ee2efe705f69 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:36.879322 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8069bfca-9f4e-49d6-9ea8-ee2efe705f69 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:36.879632 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8069bfca-9f4e-49d6-9ea8-ee2efe705f69 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:36.881008 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 26/49] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:09:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:09:36.886299 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689f2b70-8b6a-49b2-bb71-78713ce61489 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:36.886586 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689f2b70-8b6a-49b2-bb71-78713ce61489 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:36.886871 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689f2b70-8b6a-49b2-bb71-78713ce61489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:36.904114 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-689f2b70-8b6a-49b2-bb71-78713ce61489 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:09:36.905295 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-689f2b70-8b6a-49b2-bb71-78713ce61489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:09:36.926028 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b423e336-1524-4c40-a06e-41641bc6dd26 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:36.958502 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95870086-051d-4dca-8eca-09a54ec9e7e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:36.959289 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95870086-051d-4dca-8eca-09a54ec9e7e3 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:36.959601 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95870086-051d-4dca-8eca-09a54ec9e7e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:36.959874 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95870086-051d-4dca-8eca-09a54ec9e7e3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:36.967366 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 27/50] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:09:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 17 19:09:36.969739 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95870086-051d-4dca-8eca-09a54ec9e7e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 17 19:09:36.977555 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-95870086-051d-4dca-8eca-09a54ec9e7e3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 17 19:09:36.977555 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:09:36.977555 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:09:36.977555 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:09:36.977555 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:09:36.977555 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:09:36.977555 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:09:36.977555 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:09:36.977555 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:09:36.977555 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:09:36.977555 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:09:36.977555 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:09:36.977555 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 17 19:09:36.977555 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) May 17 19:09:36.977555 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 17 19:09:36.977555 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 17 19:09:36.978377 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 17 19:09:36.978377 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 17 19:09:36.978377 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 17 19:09:36.978377 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 17 19:09:36.978377 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:09:36.978377 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:09:36.978377 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:09:36.978377 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:09:36.978377 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:09:36.978377 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:09:36.978377 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:09:36.978377 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:09:36.978377 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:09:36.978377 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:09:36.978377 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:09:36.978377 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:09:36.979122 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:09:36.979122 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:09:36.979122 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:09:36.979122 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:09:36.979122 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 17 19:09:36.979122 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 17 19:09:36.979122 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 17 19:09:36.979122 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 17 19:09:36.979122 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 17 19:09:36.979122 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 17 19:09:36.979122 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 17 19:09:36.979122 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:09:36.979122 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 24/51] 10.222.0.58 () {56 vars in 1096 bytes} [Tue May 17 19:09:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:09:36.984301 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689f2b70-8b6a-49b2-bb71-78713ce61489 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:36.989659 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95870086-051d-4dca-8eca-09a54ec9e7e3 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:37.076443 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689f2b70-8b6a-49b2-bb71-78713ce61489 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:37.082640 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95870086-051d-4dca-8eca-09a54ec9e7e3 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:37.107650 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7d0964-a92a-4d0f-9b74-1c6f484509c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:37.108308 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7d0964-a92a-4d0f-9b74-1c6f484509c2 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:37.108584 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7d0964-a92a-4d0f-9b74-1c6f484509c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:37.108858 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7d0964-a92a-4d0f-9b74-1c6f484509c2 None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:37.109600 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7d0964-a92a-4d0f-9b74-1c6f484509c2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:09:37.109899 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7d0964-a92a-4d0f-9b74-1c6f484509c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:37.111288 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7d0964-a92a-4d0f-9b74-1c6f484509c2 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:37.114246 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fd392a-ea12-4f86-a755-7fcb63390b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:37.115025 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fd392a-ea12-4f86-a755-7fcb63390b8b None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:37.115307 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fd392a-ea12-4f86-a755-7fcb63390b8b None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:37.115621 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fd392a-ea12-4f86-a755-7fcb63390b8b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:37.116793 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fd392a-ea12-4f86-a755-7fcb63390b8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:37.118762 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fd392a-ea12-4f86-a755-7fcb63390b8b None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:37.120399 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 25/52] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:09:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:09:37.122910 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 28/53] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 17 19:09:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:09:37.125678 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fd392a-ea12-4f86-a755-7fcb63390b8b None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:37.127863 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7d0964-a92a-4d0f-9b74-1c6f484509c2 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:37.217984 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fd392a-ea12-4f86-a755-7fcb63390b8b None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:37.219027 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7d0964-a92a-4d0f-9b74-1c6f484509c2 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:37.250126 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c16010-e96f-4f60-b08c-65c711602acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:37.250517 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae89bd83-59fd-49cc-92a1-92565aecffd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:37.250773 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c16010-e96f-4f60-b08c-65c711602acb None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:37.251060 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c16010-e96f-4f60-b08c-65c711602acb None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:37.251338 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c16010-e96f-4f60-b08c-65c711602acb None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:37.251420 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae89bd83-59fd-49cc-92a1-92565aecffd2 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:37.251671 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae89bd83-59fd-49cc-92a1-92565aecffd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:37.251950 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae89bd83-59fd-49cc-92a1-92565aecffd2 None admin] PATH_INFO: `/v3/users/ac2441e330cd41aaa04d125461331f3b` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:37.252075 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c16010-e96f-4f60-b08c-65c711602acb None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:37.253162 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae89bd83-59fd-49cc-92a1-92565aecffd2 None admin] RBAC: Authorizing `identity:get_user(user_id=ac2441e330cd41aaa04d125461331f3b)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:37.253531 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c16010-e96f-4f60-b08c-65c711602acb None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:37.254595 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae89bd83-59fd-49cc-92a1-92565aecffd2 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:37.256424 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 29/54] 10.222.0.58 () {56 vars in 1177 bytes} [Tue May 17 19:09:37 2022] GET /identity/v3/users/ac2441e330cd41aaa04d125461331f3b => generated 247 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:09:37.261984 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae89bd83-59fd-49cc-92a1-92565aecffd2 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:37.354079 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae89bd83-59fd-49cc-92a1-92565aecffd2 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:37.360178 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 26/55] 10.222.0.58 () {60 vars in 1140 bytes} [Tue May 17 19:09:37 2022] POST /identity/v3/projects => generated 298 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 17 19:09:37.388279 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d2217c-152f-4049-9e8f-eb30162d8f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:37.389099 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d2217c-152f-4049-9e8f-eb30162d8f29 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:37.389380 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d2217c-152f-4049-9e8f-eb30162d8f29 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:37.389655 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d2217c-152f-4049-9e8f-eb30162d8f29 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:37.390922 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d2217c-152f-4049-9e8f-eb30162d8f29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:37.392459 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d2217c-152f-4049-9e8f-eb30162d8f29 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:37.394169 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 30/56] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:09:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:09:37.400927 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c16010-e96f-4f60-b08c-65c711602acb None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:37.503126 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c16010-e96f-4f60-b08c-65c711602acb None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:37.536019 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba6a0d3-c162-4b40-bd5a-4191a88fbf30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:37.536684 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba6a0d3-c162-4b40-bd5a-4191a88fbf30 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:37.536960 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba6a0d3-c162-4b40-bd5a-4191a88fbf30 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:37.537238 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba6a0d3-c162-4b40-bd5a-4191a88fbf30 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:37.538064 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba6a0d3-c162-4b40-bd5a-4191a88fbf30 None admin] RBAC: Adding query filter params (role.id=60af5088793e4379b88821a86552a344, scope.domain.id=default, user.id=ac2441e330cd41aaa04d125461331f3b) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:09:37.538370 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba6a0d3-c162-4b40-bd5a-4191a88fbf30 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:37.540348 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba6a0d3-c162-4b40-bd5a-4191a88fbf30 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:37.546036 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 27/57] 10.222.0.58 () {56 vars in 1322 bytes} [Tue May 17 19:09:37 2022] GET /identity/v3/role_assignments?user.id=ac2441e330cd41aaa04d125461331f3b&scope.domain.id=default&role.id=60af5088793e4379b88821a86552a344 => generated 530 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:09:39.558999 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a21d3f-b5c4-44c1-82dc-a1d5dd66ff94 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:39.559540 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a21d3f-b5c4-44c1-82dc-a1d5dd66ff94 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:39.559647 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a21d3f-b5c4-44c1-82dc-a1d5dd66ff94 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:39.560952 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 31/58] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:09:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:09:39.566004 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1b6482-abc0-41e8-9d61-32c8917b626f None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:39.566361 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1b6482-abc0-41e8-9d61-32c8917b626f None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:39.566568 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1b6482-abc0-41e8-9d61-32c8917b626f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:39.583240 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-fb1b6482-abc0-41e8-9d61-32c8917b626f None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:09:39.584445 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-fb1b6482-abc0-41e8-9d61-32c8917b626f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:09:39.646640 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 28/59] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:09:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 17 19:09:39.670346 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a21d3f-b5c4-44c1-82dc-a1d5dd66ff94 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:39.765831 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a21d3f-b5c4-44c1-82dc-a1d5dd66ff94 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:39.797256 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bada7a7-ba9e-45bc-b194-c88f37a2a5f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:39.798029 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bada7a7-ba9e-45bc-b194-c88f37a2a5f8 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:39.798313 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bada7a7-ba9e-45bc-b194-c88f37a2a5f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:39.798591 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bada7a7-ba9e-45bc-b194-c88f37a2a5f8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:39.799794 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bada7a7-ba9e-45bc-b194-c88f37a2a5f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:39.801277 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bada7a7-ba9e-45bc-b194-c88f37a2a5f8 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:39.802904 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 32/60] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:09:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:09:39.808593 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1b6482-abc0-41e8-9d61-32c8917b626f None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:39.902130 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1b6482-abc0-41e8-9d61-32c8917b626f None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:39.933841 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597e56df-a5f0-4122-814b-b1cbb961a4ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:39.934506 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597e56df-a5f0-4122-814b-b1cbb961a4ec None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:39.934794 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597e56df-a5f0-4122-814b-b1cbb961a4ec None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:39.935072 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597e56df-a5f0-4122-814b-b1cbb961a4ec None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:39.935801 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597e56df-a5f0-4122-814b-b1cbb961a4ec None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:39.937247 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597e56df-a5f0-4122-814b-b1cbb961a4ec None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:40.045189 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 29/61] 10.222.0.58 () {60 vars in 1132 bytes} [Tue May 17 19:09:39 2022] POST /identity/v3/users => generated 275 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:09:43.284571 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c1a504-fac0-4dba-a36c-bdd6ee16f82d None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:43.285396 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c1a504-fac0-4dba-a36c-bdd6ee16f82d None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:43.285840 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c1a504-fac0-4dba-a36c-bdd6ee16f82d None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:43.287359 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 33/62] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:09:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:09:43.293359 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139a0572-3623-4a87-b113-1d5fe1e0fbb1 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:43.294144 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139a0572-3623-4a87-b113-1d5fe1e0fbb1 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:43.294565 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139a0572-3623-4a87-b113-1d5fe1e0fbb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:43.320515 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-139a0572-3623-4a87-b113-1d5fe1e0fbb1 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:09:43.322481 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-139a0572-3623-4a87-b113-1d5fe1e0fbb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:09:43.405808 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 30/63] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:09:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 17 19:09:43.453563 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c1a504-fac0-4dba-a36c-bdd6ee16f82d None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:43.631596 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5a5e9b-456b-4a9f-a026-6ecec0ceadde None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:43.631596 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5a5e9b-456b-4a9f-a026-6ecec0ceadde None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:43.631596 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5a5e9b-456b-4a9f-a026-6ecec0ceadde None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:43.631596 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 31/64] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:09:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 17 19:09:43.632096 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c1a504-fac0-4dba-a36c-bdd6ee16f82d None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:43.632195 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07a813f-396d-4305-b406-719ad0585a03 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:43.632195 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07a813f-396d-4305-b406-719ad0585a03 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:43.632195 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07a813f-396d-4305-b406-719ad0585a03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:43.637629 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-b07a813f-396d-4305-b406-719ad0585a03 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:09:43.637765 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06569470-cb39-4419-b036-39a5c123313b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:43.638713 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06569470-cb39-4419-b036-39a5c123313b None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:43.638822 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b07a813f-396d-4305-b406-719ad0585a03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:09:43.639245 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06569470-cb39-4419-b036-39a5c123313b None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:43.639699 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06569470-cb39-4419-b036-39a5c123313b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:43.652846 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06569470-cb39-4419-b036-39a5c123313b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 17 19:09:43.663196 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-06569470-cb39-4419-b036-39a5c123313b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 17 19:09:43.663196 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:09:43.663196 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:09:43.663196 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:09:43.663196 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:09:43.663196 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:09:43.663196 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:09:43.663196 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:09:43.663196 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:09:43.663196 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:09:43.663196 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:09:43.663196 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:09:43.663196 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 17 19:09:43.663196 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) May 17 19:09:43.663196 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 17 19:09:43.663196 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 17 19:09:43.664202 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 17 19:09:43.664202 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 17 19:09:43.664202 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 17 19:09:43.664202 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 17 19:09:43.664202 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:09:43.664202 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:09:43.664202 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:09:43.664202 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:09:43.664202 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:09:43.664202 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:09:43.664202 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:09:43.664202 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:09:43.664202 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:09:43.664202 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:09:43.664202 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:09:43.664202 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:09:43.665882 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:09:43.665882 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:09:43.665882 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:09:43.665882 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:09:43.665882 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 17 19:09:43.665882 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 17 19:09:43.665882 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 17 19:09:43.665882 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 17 19:09:43.665882 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 17 19:09:43.665882 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 17 19:09:43.665882 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 17 19:09:43.665882 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:09:43.668356 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 34/65] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 17 19:09:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:09:43.670069 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ed9533-3ee3-40ee-946b-c38c7c59ccb3 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:43.670554 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ed9533-3ee3-40ee-946b-c38c7c59ccb3 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:43.672485 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ed9533-3ee3-40ee-946b-c38c7c59ccb3 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:43.674215 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 35/66] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:09:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:09:43.675152 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ed9533-3ee3-40ee-946b-c38c7c59ccb3 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:43.724760 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 32/67] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:09:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 17 19:09:43.726475 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c05bd94-1057-48cc-a3d1-0e6950097321 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:43.727532 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c05bd94-1057-48cc-a3d1-0e6950097321 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:43.728013 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c05bd94-1057-48cc-a3d1-0e6950097321 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:43.743882 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-5c05bd94-1057-48cc-a3d1-0e6950097321 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:09:43.745331 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-5c05bd94-1057-48cc-a3d1-0e6950097321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:09:43.785470 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ed9533-3ee3-40ee-946b-c38c7c59ccb3 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:43.810881 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 33/68] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:09:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 17 19:09:43.813110 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572663a7-87ab-45df-aa3a-b56f4f729fa8 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:43.813406 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572663a7-87ab-45df-aa3a-b56f4f729fa8 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:43.813686 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572663a7-87ab-45df-aa3a-b56f4f729fa8 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:43.815093 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 34/69] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:09:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:09:43.815889 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572663a7-87ab-45df-aa3a-b56f4f729fa8 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:43.823147 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6335cf-6180-4970-8237-e78f1e5660c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:43.824187 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6335cf-6180-4970-8237-e78f1e5660c0 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:43.824675 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6335cf-6180-4970-8237-e78f1e5660c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:43.825157 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6335cf-6180-4970-8237-e78f1e5660c0 None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:43.826536 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6335cf-6180-4970-8237-e78f1e5660c0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:09:43.827051 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6335cf-6180-4970-8237-e78f1e5660c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:43.829326 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6335cf-6180-4970-8237-e78f1e5660c0 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:43.840460 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 36/70] 10.222.0.58 () {56 vars in 1101 bytes} [Tue May 17 19:09:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:09:43.842429 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f61229-ecb8-430a-b173-f336093648a8 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:43.843250 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f61229-ecb8-430a-b173-f336093648a8 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:43.844058 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f61229-ecb8-430a-b173-f336093648a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:43.862715 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-b8f61229-ecb8-430a-b173-f336093648a8 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:09:43.863939 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-b8f61229-ecb8-430a-b173-f336093648a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:09:43.920550 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572663a7-87ab-45df-aa3a-b56f4f729fa8 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:43.943516 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 37/71] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:09:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 17 19:09:43.943516 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f61229-ecb8-430a-b173-f336093648a8 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:43.970029 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8835387c-eb1c-4942-a3a9-cb1dbb28ccb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:43.970827 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8835387c-eb1c-4942-a3a9-cb1dbb28ccb5 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:43.971109 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8835387c-eb1c-4942-a3a9-cb1dbb28ccb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:43.971375 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8835387c-eb1c-4942-a3a9-cb1dbb28ccb5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:43.982646 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8835387c-eb1c-4942-a3a9-cb1dbb28ccb5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 17 19:09:43.990227 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-8835387c-eb1c-4942-a3a9-cb1dbb28ccb5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 17 19:09:43.990227 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:09:43.990227 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:09:43.990227 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:09:43.990227 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:09:43.990227 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:09:43.990227 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:09:43.990227 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:09:43.990227 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:09:43.990227 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:09:43.990227 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:09:43.990227 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:09:43.990227 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 17 19:09:43.990227 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) May 17 19:09:43.990227 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 17 19:09:43.990227 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 17 19:09:43.991088 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 17 19:09:43.991088 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 17 19:09:43.991088 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 17 19:09:43.991088 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 17 19:09:43.991088 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:09:43.991088 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:09:43.991088 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:09:43.991088 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:09:43.991088 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:09:43.991088 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:09:43.991088 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:09:43.991088 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:09:43.991088 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:09:43.991088 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:09:43.991088 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:09:43.991088 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:09:43.991887 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:09:43.991887 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:09:43.991887 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:09:43.991887 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:09:43.991887 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 17 19:09:43.991887 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 17 19:09:43.991887 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 17 19:09:43.991887 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 17 19:09:43.991887 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 17 19:09:43.991887 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 17 19:09:43.991887 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 17 19:09:43.991887 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:09:43.991887 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 35/72] 10.222.0.58 () {56 vars in 1096 bytes} [Tue May 17 19:09:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:09:43.992552 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8835387c-eb1c-4942-a3a9-cb1dbb28ccb5 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:44.054448 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f61229-ecb8-430a-b173-f336093648a8 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:44.091449 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94f5a3f-2395-4e7c-bd77-4d2f6ab40132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:44.092274 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94f5a3f-2395-4e7c-bd77-4d2f6ab40132 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:44.092557 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94f5a3f-2395-4e7c-bd77-4d2f6ab40132 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:44.092711 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8835387c-eb1c-4942-a3a9-cb1dbb28ccb5 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:44.092869 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94f5a3f-2395-4e7c-bd77-4d2f6ab40132 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:44.103188 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94f5a3f-2395-4e7c-bd77-4d2f6ab40132 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 17 19:09:44.110815 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-a94f5a3f-2395-4e7c-bd77-4d2f6ab40132 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 17 19:09:44.110815 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:09:44.110815 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:09:44.110815 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:09:44.110815 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:09:44.110815 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:09:44.110815 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:09:44.110815 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:09:44.110815 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:09:44.110815 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:09:44.110815 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:09:44.110815 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:09:44.110815 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 17 19:09:44.110815 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) May 17 19:09:44.110815 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 17 19:09:44.110815 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 17 19:09:44.111942 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 17 19:09:44.111942 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 17 19:09:44.111942 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 17 19:09:44.111942 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 17 19:09:44.111942 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:09:44.111942 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:09:44.111942 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:09:44.111942 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:09:44.111942 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:09:44.111942 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:09:44.111942 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:09:44.111942 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:09:44.111942 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:09:44.111942 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:09:44.111942 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:09:44.111942 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:09:44.112712 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:09:44.112712 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:09:44.112712 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:09:44.112712 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:09:44.112712 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 17 19:09:44.112712 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 17 19:09:44.112712 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 17 19:09:44.112712 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 17 19:09:44.112712 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 17 19:09:44.112712 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 17 19:09:44.112712 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 17 19:09:44.112712 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:09:44.112712 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 38/73] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 17 19:09:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:09:44.113321 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94f5a3f-2395-4e7c-bd77-4d2f6ab40132 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:44.128428 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428f65eb-85df-4a6a-8a69-62844a3d6516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:44.129303 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428f65eb-85df-4a6a-8a69-62844a3d6516 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:44.129587 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428f65eb-85df-4a6a-8a69-62844a3d6516 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:44.129866 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428f65eb-85df-4a6a-8a69-62844a3d6516 None admin] PATH_INFO: `/v3/users/ed3e18e472cf4c8fb0e0f8e477b961b1` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:44.143604 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428f65eb-85df-4a6a-8a69-62844a3d6516 None admin] RBAC: Authorizing `identity:get_user(user_id=ed3e18e472cf4c8fb0e0f8e477b961b1)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:44.145090 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428f65eb-85df-4a6a-8a69-62844a3d6516 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:44.151886 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 36/74] 10.222.0.58 () {56 vars in 1177 bytes} [Tue May 17 19:09:43 2022] GET /identity/v3/users/ed3e18e472cf4c8fb0e0f8e477b961b1 => generated 275 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:09:44.151886 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428f65eb-85df-4a6a-8a69-62844a3d6516 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:44.219975 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94f5a3f-2395-4e7c-bd77-4d2f6ab40132 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:44.243555 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428f65eb-85df-4a6a-8a69-62844a3d6516 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:44.252558 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96a843a-d7e4-47a9-9b16-916246ceb098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:44.253364 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96a843a-d7e4-47a9-9b16-916246ceb098 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:44.253650 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96a843a-d7e4-47a9-9b16-916246ceb098 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:44.253927 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96a843a-d7e4-47a9-9b16-916246ceb098 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:44.263565 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96a843a-d7e4-47a9-9b16-916246ceb098 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 17 19:09:44.271369 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-d96a843a-d7e4-47a9-9b16-916246ceb098 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 17 19:09:44.271369 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:09:44.271369 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:09:44.271369 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:09:44.271369 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:09:44.271369 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:09:44.271369 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:09:44.271369 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:09:44.271369 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:09:44.271369 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:09:44.271369 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:09:44.271369 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:09:44.271369 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 17 19:09:44.271369 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) May 17 19:09:44.271369 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 17 19:09:44.271369 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 17 19:09:44.272262 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 17 19:09:44.272262 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 17 19:09:44.272262 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 17 19:09:44.272262 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 17 19:09:44.272262 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:09:44.272262 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:09:44.272262 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:09:44.272262 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:09:44.272262 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:09:44.272262 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:09:44.272262 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:09:44.272262 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:09:44.272262 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:09:44.272262 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:09:44.272262 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:09:44.272262 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:09:44.273086 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:09:44.273086 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:09:44.273086 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:09:44.273086 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:09:44.273086 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 17 19:09:44.273086 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 17 19:09:44.273086 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 17 19:09:44.273086 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 17 19:09:44.273086 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 17 19:09:44.273086 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 17 19:09:44.273086 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 17 19:09:44.273086 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:09:44.273086 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 39/75] 10.222.0.58 () {56 vars in 1114 bytes} [Tue May 17 19:09:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:09:44.273771 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96a843a-d7e4-47a9-9b16-916246ceb098 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:44.276183 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577f5c6e-4b31-4cf1-9e26-7da5f31e1eb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:44.276863 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577f5c6e-4b31-4cf1-9e26-7da5f31e1eb3 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:44.277151 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577f5c6e-4b31-4cf1-9e26-7da5f31e1eb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:44.277433 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577f5c6e-4b31-4cf1-9e26-7da5f31e1eb3 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:44.278178 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577f5c6e-4b31-4cf1-9e26-7da5f31e1eb3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:09:44.279301 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577f5c6e-4b31-4cf1-9e26-7da5f31e1eb3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:44.280185 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577f5c6e-4b31-4cf1-9e26-7da5f31e1eb3 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:44.288234 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 37/76] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 17 19:09:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:09:44.289016 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577f5c6e-4b31-4cf1-9e26-7da5f31e1eb3 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:44.369134 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96a843a-d7e4-47a9-9b16-916246ceb098 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:44.385505 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577f5c6e-4b31-4cf1-9e26-7da5f31e1eb3 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:44.401020 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78966196-5027-48ba-9556-35151824ad8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:44.401683 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78966196-5027-48ba-9556-35151824ad8c None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:44.401970 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78966196-5027-48ba-9556-35151824ad8c None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:44.402247 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78966196-5027-48ba-9556-35151824ad8c None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:44.402990 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78966196-5027-48ba-9556-35151824ad8c None admin] RBAC: Adding query filter params (name=member) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:09:44.403296 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78966196-5027-48ba-9556-35151824ad8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:44.404711 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78966196-5027-48ba-9556-35151824ad8c None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:44.412375 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 40/77] 10.222.0.58 () {56 vars in 1101 bytes} [Tue May 17 19:09:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:09:44.417517 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bc4ccf-c625-4ea5-b287-94c80254392e None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:44.417517 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bc4ccf-c625-4ea5-b287-94c80254392e None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:44.417517 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bc4ccf-c625-4ea5-b287-94c80254392e None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:44.417517 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 41/78] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:09:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:09:44.417517 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bc4ccf-c625-4ea5-b287-94c80254392e None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:44.421332 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c1d41f-e6c6-4fa7-b636-66dcce34e8a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:44.421983 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c1d41f-e6c6-4fa7-b636-66dcce34e8a4 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:44.422268 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c1d41f-e6c6-4fa7-b636-66dcce34e8a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:44.422546 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c1d41f-e6c6-4fa7-b636-66dcce34e8a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:44.486700 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c1d41f-e6c6-4fa7-b636-66dcce34e8a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:44.488161 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c1d41f-e6c6-4fa7-b636-66dcce34e8a4 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:44.512045 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bc4ccf-c625-4ea5-b287-94c80254392e None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:44.543858 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7fd5b9-97d4-4bd1-a16f-5bcdb64a14d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:44.544509 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7fd5b9-97d4-4bd1-a16f-5bcdb64a14d5 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:44.544796 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7fd5b9-97d4-4bd1-a16f-5bcdb64a14d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:44.545074 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7fd5b9-97d4-4bd1-a16f-5bcdb64a14d5 None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:44.545815 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7fd5b9-97d4-4bd1-a16f-5bcdb64a14d5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:09:44.546124 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7fd5b9-97d4-4bd1-a16f-5bcdb64a14d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:44.547523 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7fd5b9-97d4-4bd1-a16f-5bcdb64a14d5 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:44.555152 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 42/79] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 17 19:09:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:09:44.557078 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7fd5b9-97d4-4bd1-a16f-5bcdb64a14d5 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:44.579687 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 38/80] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 17 19:09:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 17 19:09:44.580515 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c1d41f-e6c6-4fa7-b636-66dcce34e8a4 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:44.649927 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7fd5b9-97d4-4bd1-a16f-5bcdb64a14d5 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:44.673063 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c1d41f-e6c6-4fa7-b636-66dcce34e8a4 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:44.681140 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c483d38c-c7cf-4aee-ae71-12b29fd71c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:44.682002 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c483d38c-c7cf-4aee-ae71-12b29fd71c3f None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:44.682284 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c483d38c-c7cf-4aee-ae71-12b29fd71c3f None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:44.682563 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c483d38c-c7cf-4aee-ae71-12b29fd71c3f None admin] PATH_INFO: `/v3/users/ac2441e330cd41aaa04d125461331f3b` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:44.683784 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c483d38c-c7cf-4aee-ae71-12b29fd71c3f None admin] RBAC: Authorizing `identity:get_user(user_id=ac2441e330cd41aaa04d125461331f3b)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:44.685223 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c483d38c-c7cf-4aee-ae71-12b29fd71c3f None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:44.686838 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 43/81] 10.222.0.58 () {56 vars in 1177 bytes} [Tue May 17 19:09:44 2022] GET /identity/v3/users/ac2441e330cd41aaa04d125461331f3b => generated 247 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:09:44.687979 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf1c7de-410e-44f5-9288-d002e5149253 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:44.688263 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf1c7de-410e-44f5-9288-d002e5149253 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:44.688544 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf1c7de-410e-44f5-9288-d002e5149253 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:44.703876 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-9cf1c7de-410e-44f5-9288-d002e5149253 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:09:44.704833 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1c5fb7-3747-4af3-813a-93936a136309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:44.705063 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-9cf1c7de-410e-44f5-9288-d002e5149253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:09:44.705697 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1c5fb7-3747-4af3-813a-93936a136309 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:44.705975 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1c5fb7-3747-4af3-813a-93936a136309 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:44.706246 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1c5fb7-3747-4af3-813a-93936a136309 None admin] PATH_INFO: `/v3/users/ed3e18e472cf4c8fb0e0f8e477b961b1` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:44.707422 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1c5fb7-3747-4af3-813a-93936a136309 None admin] RBAC: Authorizing `identity:get_user(user_id=ed3e18e472cf4c8fb0e0f8e477b961b1)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:44.708903 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1c5fb7-3747-4af3-813a-93936a136309 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:44.710702 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 39/82] 10.222.0.58 () {56 vars in 1177 bytes} [Tue May 17 19:09:44 2022] GET /identity/v3/users/ed3e18e472cf4c8fb0e0f8e477b961b1 => generated 275 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:09:44.711767 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1c5fb7-3747-4af3-813a-93936a136309 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:44.767214 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 44/83] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:09:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 17 19:09:44.768000 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf1c7de-410e-44f5-9288-d002e5149253 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:44.806749 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1c5fb7-3747-4af3-813a-93936a136309 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:44.839359 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d86842e-6b89-4a48-8c4c-51016626f8bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:44.840241 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d86842e-6b89-4a48-8c4c-51016626f8bc None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:44.840522 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d86842e-6b89-4a48-8c4c-51016626f8bc None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:44.840795 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d86842e-6b89-4a48-8c4c-51016626f8bc None admin] PATH_INFO: `/v3/users/ed3e18e472cf4c8fb0e0f8e477b961b1` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:44.841977 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d86842e-6b89-4a48-8c4c-51016626f8bc None admin] RBAC: Authorizing `identity:get_user(user_id=ed3e18e472cf4c8fb0e0f8e477b961b1)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:44.843441 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d86842e-6b89-4a48-8c4c-51016626f8bc None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:44.845095 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 40/84] 10.222.0.58 () {56 vars in 1177 bytes} [Tue May 17 19:09:44 2022] GET /identity/v3/users/ed3e18e472cf4c8fb0e0f8e477b961b1 => generated 275 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:09:44.845855 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d86842e-6b89-4a48-8c4c-51016626f8bc None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:44.864579 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf1c7de-410e-44f5-9288-d002e5149253 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:44.896790 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10a99c0-421a-46d1-8a6d-dd2d1dddf4d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:44.897639 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10a99c0-421a-46d1-8a6d-dd2d1dddf4d7 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:44.897920 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10a99c0-421a-46d1-8a6d-dd2d1dddf4d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:44.898196 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10a99c0-421a-46d1-8a6d-dd2d1dddf4d7 None admin] PATH_INFO: `/v3/projects/d13e273646b44ecca3cd10cd867bbd04` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:44.899970 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10a99c0-421a-46d1-8a6d-dd2d1dddf4d7 None admin] RBAC: Authorizing `identity:get_project(project_id=d13e273646b44ecca3cd10cd867bbd04)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:44.901413 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10a99c0-421a-46d1-8a6d-dd2d1dddf4d7 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:44.903111 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 45/85] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 17 19:09:44 2022] GET /identity/v3/projects/d13e273646b44ecca3cd10cd867bbd04 => generated 298 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:09:44.903902 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10a99c0-421a-46d1-8a6d-dd2d1dddf4d7 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:44.939263 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d86842e-6b89-4a48-8c4c-51016626f8bc None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:44.972735 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b846e34-4f13-47fe-9278-bb40d4184a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:44.973380 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b846e34-4f13-47fe-9278-bb40d4184a0f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:44.973664 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b846e34-4f13-47fe-9278-bb40d4184a0f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:44.973947 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b846e34-4f13-47fe-9278-bb40d4184a0f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:44.999291 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10a99c0-421a-46d1-8a6d-dd2d1dddf4d7 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:45.031558 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69c1ae0-e593-4dd2-bdba-99f6a4693b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:45.032148 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69c1ae0-e593-4dd2-bdba-99f6a4693b8e None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:45.032432 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69c1ae0-e593-4dd2-bdba-99f6a4693b8e None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:45.032714 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69c1ae0-e593-4dd2-bdba-99f6a4693b8e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:45.038086 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b846e34-4f13-47fe-9278-bb40d4184a0f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:45.039689 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b846e34-4f13-47fe-9278-bb40d4184a0f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:45.095498 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69c1ae0-e593-4dd2-bdba-99f6a4693b8e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:45.097245 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69c1ae0-e593-4dd2-bdba-99f6a4693b8e None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:45.132424 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 41/86] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 17 19:09:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 17 19:09:45.133780 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b846e34-4f13-47fe-9278-bb40d4184a0f None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:45.189460 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 46/87] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 17 19:09:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 17 19:09:45.192433 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69c1ae0-e593-4dd2-bdba-99f6a4693b8e None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:45.230782 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b846e34-4f13-47fe-9278-bb40d4184a0f None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:45.263430 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a6108a-4a87-4f40-a5a4-04e1d9c8a24e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:45.264239 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a6108a-4a87-4f40-a5a4-04e1d9c8a24e None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:45.264517 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a6108a-4a87-4f40-a5a4-04e1d9c8a24e None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:45.264782 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a6108a-4a87-4f40-a5a4-04e1d9c8a24e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:45.265942 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a6108a-4a87-4f40-a5a4-04e1d9c8a24e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:45.267378 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a6108a-4a87-4f40-a5a4-04e1d9c8a24e None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:45.268996 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 42/88] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:09:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:09:45.269751 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a6108a-4a87-4f40-a5a4-04e1d9c8a24e None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:45.285093 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69c1ae0-e593-4dd2-bdba-99f6a4693b8e None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:45.316237 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030289fe-aff8-40b0-8a24-23f7d0b1687c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:45.316873 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030289fe-aff8-40b0-8a24-23f7d0b1687c None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:45.317156 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030289fe-aff8-40b0-8a24-23f7d0b1687c None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:45.317425 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030289fe-aff8-40b0-8a24-23f7d0b1687c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:45.362980 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a6108a-4a87-4f40-a5a4-04e1d9c8a24e None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:45.380351 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030289fe-aff8-40b0-8a24-23f7d0b1687c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:45.381809 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030289fe-aff8-40b0-8a24-23f7d0b1687c None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:45.395960 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6357379-d3cf-4c82-93e4-d483a9338de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:45.396619 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6357379-d3cf-4c82-93e4-d483a9338de5 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:45.396903 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6357379-d3cf-4c82-93e4-d483a9338de5 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:45.397178 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6357379-d3cf-4c82-93e4-d483a9338de5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:45.397998 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6357379-d3cf-4c82-93e4-d483a9338de5 None admin] RBAC: Adding query filter params (role.id=114e6a16e63447d1924707ace5aace25, scope.project.id=d13e273646b44ecca3cd10cd867bbd04, user.id=ed3e18e472cf4c8fb0e0f8e477b961b1) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:09:45.398305 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6357379-d3cf-4c82-93e4-d483a9338de5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:45.399780 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6357379-d3cf-4c82-93e4-d483a9338de5 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:45.405014 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 43/89] 10.222.0.58 () {56 vars in 1374 bytes} [Tue May 17 19:09:45 2022] GET /identity/v3/role_assignments?user.id=ed3e18e472cf4c8fb0e0f8e477b961b1&scope.project.id=d13e273646b44ecca3cd10cd867bbd04&role.id=114e6a16e63447d1924707ace5aace25 => generated 259 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:09:45.405777 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6357379-d3cf-4c82-93e4-d483a9338de5 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:45.483798 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 47/90] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 17 19:09:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 17 19:09:45.485056 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030289fe-aff8-40b0-8a24-23f7d0b1687c None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:45.511458 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6357379-d3cf-4c82-93e4-d483a9338de5 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:45.546863 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d156cf-0210-4e6f-b646-9c8aae3c8202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:45.549394 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d156cf-0210-4e6f-b646-9c8aae3c8202 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:45.549394 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d156cf-0210-4e6f-b646-9c8aae3c8202 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:45.549394 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d156cf-0210-4e6f-b646-9c8aae3c8202 None admin] PATH_INFO: `/v3/projects/d13e273646b44ecca3cd10cd867bbd04` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:45.550546 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d156cf-0210-4e6f-b646-9c8aae3c8202 None admin] RBAC: Authorizing `identity:get_project(project_id=d13e273646b44ecca3cd10cd867bbd04)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:45.552064 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d156cf-0210-4e6f-b646-9c8aae3c8202 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:45.553866 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 44/91] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 17 19:09:45 2022] GET /identity/v3/projects/d13e273646b44ecca3cd10cd867bbd04 => generated 298 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:09:45.559550 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d156cf-0210-4e6f-b646-9c8aae3c8202 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:45.588358 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030289fe-aff8-40b0-8a24-23f7d0b1687c None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:45.622754 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d78fc02-4a6f-4908-a4e1-83559da91664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:45.623454 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d78fc02-4a6f-4908-a4e1-83559da91664 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:45.623764 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d78fc02-4a6f-4908-a4e1-83559da91664 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:45.624040 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d78fc02-4a6f-4908-a4e1-83559da91664 None admin] PATH_INFO: `/v3/projects/e2de3fd8bde6430aa6a1e97114f6aa87` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:45.625267 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d78fc02-4a6f-4908-a4e1-83559da91664 None admin] RBAC: Authorizing `identity:get_project(project_id=e2de3fd8bde6430aa6a1e97114f6aa87)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:45.626709 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d78fc02-4a6f-4908-a4e1-83559da91664 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:45.628643 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 48/92] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 17 19:09:45 2022] GET /identity/v3/projects/e2de3fd8bde6430aa6a1e97114f6aa87 => generated 312 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:09:45.629188 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d78fc02-4a6f-4908-a4e1-83559da91664 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:45.661262 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d156cf-0210-4e6f-b646-9c8aae3c8202 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:45.695280 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2435f2a-61dd-45c0-a5bf-5cc1180a85d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:45.696301 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2435f2a-61dd-45c0-a5bf-5cc1180a85d1 None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:45.696584 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2435f2a-61dd-45c0-a5bf-5cc1180a85d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:45.696912 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2435f2a-61dd-45c0-a5bf-5cc1180a85d1 None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:45.697613 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2435f2a-61dd-45c0-a5bf-5cc1180a85d1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:45.699065 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2435f2a-61dd-45c0-a5bf-5cc1180a85d1 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:45.723836 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d78fc02-4a6f-4908-a4e1-83559da91664 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:45.756596 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2e221f-f060-430b-8f6c-e0ba295ff7ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:45.757895 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2e221f-f060-430b-8f6c-e0ba295ff7ee None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:45.757895 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2e221f-f060-430b-8f6c-e0ba295ff7ee None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:45.758125 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2e221f-f060-430b-8f6c-e0ba295ff7ee None admin] PATH_INFO: `/v3/projects/d13e273646b44ecca3cd10cd867bbd04` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:45.759284 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2e221f-f060-430b-8f6c-e0ba295ff7ee None admin] RBAC: Authorizing `identity:get_project(project_id=d13e273646b44ecca3cd10cd867bbd04)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:45.760748 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2e221f-f060-430b-8f6c-e0ba295ff7ee None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:45.762413 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 49/93] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 17 19:09:45 2022] GET /identity/v3/projects/d13e273646b44ecca3cd10cd867bbd04 => generated 298 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:09:45.763167 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2e221f-f060-430b-8f6c-e0ba295ff7ee None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:45.781795 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 45/94] 10.222.0.58 () {60 vars in 1132 bytes} [Tue May 17 19:09:45 2022] POST /identity/v3/users => generated 289 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:09:45.782911 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2435f2a-61dd-45c0-a5bf-5cc1180a85d1 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:45.873776 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2e221f-f060-430b-8f6c-e0ba295ff7ee None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:45.897303 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2435f2a-61dd-45c0-a5bf-5cc1180a85d1 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:45.912112 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799c36a1-3c48-4384-8878-4e007626499a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:45.912762 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799c36a1-3c48-4384-8878-4e007626499a None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:45.913049 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799c36a1-3c48-4384-8878-4e007626499a None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:45.913332 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799c36a1-3c48-4384-8878-4e007626499a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:45.914809 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799c36a1-3c48-4384-8878-4e007626499a None admin] RBAC: Adding query filter params (role.id=60af5088793e4379b88821a86552a344, scope.project.id=d13e273646b44ecca3cd10cd867bbd04, user.id=ac2441e330cd41aaa04d125461331f3b) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:09:45.915131 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799c36a1-3c48-4384-8878-4e007626499a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:45.916685 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799c36a1-3c48-4384-8878-4e007626499a None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:45.931533 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 50/95] 10.222.0.58 () {56 vars in 1374 bytes} [Tue May 17 19:09:45 2022] GET /identity/v3/role_assignments?user.id=ac2441e330cd41aaa04d125461331f3b&scope.project.id=d13e273646b44ecca3cd10cd867bbd04&role.id=60af5088793e4379b88821a86552a344 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:09:45.931533 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799c36a1-3c48-4384-8878-4e007626499a None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:45.968356 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61d8627-60bf-4651-bdd4-b987ab3b0433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:45.969411 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61d8627-60bf-4651-bdd4-b987ab3b0433 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:45.969899 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61d8627-60bf-4651-bdd4-b987ab3b0433 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:45.970400 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61d8627-60bf-4651-bdd4-b987ab3b0433 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:45.971452 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61d8627-60bf-4651-bdd4-b987ab3b0433 None admin] RBAC: Adding query filter params (role.id=114e6a16e63447d1924707ace5aace25, scope.project.id=e2de3fd8bde6430aa6a1e97114f6aa87, user.id=ed3e18e472cf4c8fb0e0f8e477b961b1) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:09:45.972240 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61d8627-60bf-4651-bdd4-b987ab3b0433 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:45.973917 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61d8627-60bf-4651-bdd4-b987ab3b0433 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:45.985287 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 46/96] 10.222.0.58 () {56 vars in 1374 bytes} [Tue May 17 19:09:45 2022] GET /identity/v3/role_assignments?user.id=ed3e18e472cf4c8fb0e0f8e477b961b1&scope.project.id=e2de3fd8bde6430aa6a1e97114f6aa87&role.id=114e6a16e63447d1924707ace5aace25 => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:09:46.074884 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799c36a1-3c48-4384-8878-4e007626499a None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:46.115817 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95817e8b-9a78-4988-9982-a3ba9234bc76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:46.116507 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95817e8b-9a78-4988-9982-a3ba9234bc76 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:46.116793 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95817e8b-9a78-4988-9982-a3ba9234bc76 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:46.117070 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95817e8b-9a78-4988-9982-a3ba9234bc76 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:46.117991 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95817e8b-9a78-4988-9982-a3ba9234bc76 None admin] RBAC: Adding query filter params (role.id=1055a3235b0143e58f1090c1e154e5c1, scope.project.id=d13e273646b44ecca3cd10cd867bbd04, user.id=ed3e18e472cf4c8fb0e0f8e477b961b1) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:09:46.118306 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95817e8b-9a78-4988-9982-a3ba9234bc76 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:46.119863 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95817e8b-9a78-4988-9982-a3ba9234bc76 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:46.126716 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 51/97] 10.222.0.58 () {56 vars in 1374 bytes} [Tue May 17 19:09:45 2022] GET /identity/v3/role_assignments?user.id=ed3e18e472cf4c8fb0e0f8e477b961b1&scope.project.id=d13e273646b44ecca3cd10cd867bbd04&role.id=1055a3235b0143e58f1090c1e154e5c1 => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:09:47.973733 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99071856-a1c3-4a3b-9c2b-a4f6e7f8fb37 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:47.974845 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99071856-a1c3-4a3b-9c2b-a4f6e7f8fb37 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:47.975324 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99071856-a1c3-4a3b-9c2b-a4f6e7f8fb37 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:47.977722 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 47/98] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:09:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:09:47.983173 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46a0f60-187a-4c2d-983c-3c91fd005c4b None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:47.983624 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46a0f60-187a-4c2d-983c-3c91fd005c4b None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:47.983792 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46a0f60-187a-4c2d-983c-3c91fd005c4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:48.008032 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-d46a0f60-187a-4c2d-983c-3c91fd005c4b None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:09:48.009302 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-d46a0f60-187a-4c2d-983c-3c91fd005c4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:09:48.093754 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 52/99] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:09:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 17 19:09:48.139029 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99071856-a1c3-4a3b-9c2b-a4f6e7f8fb37 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:48.269279 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99071856-a1c3-4a3b-9c2b-a4f6e7f8fb37 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:48.313337 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271ad073-061f-45cb-8618-35b04186c9be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:48.314491 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271ad073-061f-45cb-8618-35b04186c9be None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:48.314994 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271ad073-061f-45cb-8618-35b04186c9be None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:48.315506 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271ad073-061f-45cb-8618-35b04186c9be None admin] PATH_INFO: `/v3/roles/member` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:48.333488 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271ad073-061f-45cb-8618-35b04186c9be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 17 19:09:48.342530 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-271ad073-061f-45cb-8618-35b04186c9be None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 17 19:09:48.342530 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:09:48.342530 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:09:48.342530 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:09:48.342530 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:09:48.342530 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:09:48.342530 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:09:48.342530 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:09:48.342530 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:09:48.342530 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:09:48.342530 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:09:48.342530 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:09:48.342530 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 17 19:09:48.342530 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) May 17 19:09:48.342530 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 17 19:09:48.342530 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 17 19:09:48.343561 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 17 19:09:48.343561 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 17 19:09:48.343561 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 17 19:09:48.343561 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 17 19:09:48.343561 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:09:48.343561 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:09:48.343561 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:09:48.343561 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:09:48.343561 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:09:48.343561 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:09:48.343561 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:09:48.343561 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:09:48.343561 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:09:48.343561 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:09:48.343561 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:09:48.343561 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:09:48.344318 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:09:48.344318 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:09:48.344318 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:09:48.344318 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:09:48.344318 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 17 19:09:48.344318 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 17 19:09:48.344318 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 17 19:09:48.344318 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 17 19:09:48.344318 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 17 19:09:48.344318 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 17 19:09:48.344318 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 17 19:09:48.344318 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:09:48.346842 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 48/100] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 17 19:09:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:09:48.353945 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46a0f60-187a-4c2d-983c-3c91fd005c4b None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:48.471584 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46a0f60-187a-4c2d-983c-3c91fd005c4b None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:48.510956 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d642622-4076-445f-b9ee-8c8572f5f690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:48.511660 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d642622-4076-445f-b9ee-8c8572f5f690 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:48.511943 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d642622-4076-445f-b9ee-8c8572f5f690 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:48.512217 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d642622-4076-445f-b9ee-8c8572f5f690 None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:48.512973 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d642622-4076-445f-b9ee-8c8572f5f690 None admin] RBAC: Adding query filter params (name=member) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:09:48.513277 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d642622-4076-445f-b9ee-8c8572f5f690 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:48.514654 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d642622-4076-445f-b9ee-8c8572f5f690 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:48.527670 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 53/101] 10.222.0.58 () {56 vars in 1101 bytes} [Tue May 17 19:09:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:09:48.529665 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271ad073-061f-45cb-8618-35b04186c9be None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:48.647193 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb85eed-0068-46e2-86f2-533a6fe01f98 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:48.647529 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb85eed-0068-46e2-86f2-533a6fe01f98 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:48.647805 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb85eed-0068-46e2-86f2-533a6fe01f98 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:48.649931 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 54/102] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:09:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 17 19:09:48.665304 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c732e35-b46e-469c-af3d-f765436270bd None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:48.665304 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c732e35-b46e-469c-af3d-f765436270bd None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:48.665510 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c732e35-b46e-469c-af3d-f765436270bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:48.677604 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271ad073-061f-45cb-8618-35b04186c9be None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:48.704680 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-6c732e35-b46e-469c-af3d-f765436270bd None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:09:48.705880 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-6c732e35-b46e-469c-af3d-f765436270bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:09:48.721619 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c15dbac-561e-450c-8e60-6f40c13c3a3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:48.722763 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c15dbac-561e-450c-8e60-6f40c13c3a3e None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:48.725755 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c15dbac-561e-450c-8e60-6f40c13c3a3e None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:48.725755 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c15dbac-561e-450c-8e60-6f40c13c3a3e None admin] PATH_INFO: `/v3/users/ed3e18e472cf4c8fb0e0f8e477b961b1` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:48.728250 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c15dbac-561e-450c-8e60-6f40c13c3a3e None admin] RBAC: Authorizing `identity:get_user(user_id=ed3e18e472cf4c8fb0e0f8e477b961b1)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:48.729811 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c15dbac-561e-450c-8e60-6f40c13c3a3e None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:48.731671 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 49/103] 10.222.0.58 () {56 vars in 1177 bytes} [Tue May 17 19:09:48 2022] GET /identity/v3/users/ed3e18e472cf4c8fb0e0f8e477b961b1 => generated 275 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:09:48.745841 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c15dbac-561e-450c-8e60-6f40c13c3a3e None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:48.788735 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 55/104] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:09:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 17 19:09:48.824020 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c732e35-b46e-469c-af3d-f765436270bd None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:48.913862 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c15dbac-561e-450c-8e60-6f40c13c3a3e None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:48.955480 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c732e35-b46e-469c-af3d-f765436270bd None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:48.992084 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86406987-a619-4515-9504-c89c517ba0e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:48.993956 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f63fe4d-4f66-4060-9301-30ccdcaf43dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:48.995264 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f63fe4d-4f66-4060-9301-30ccdcaf43dd None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:48.996380 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86406987-a619-4515-9504-c89c517ba0e0 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:48.996882 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f63fe4d-4f66-4060-9301-30ccdcaf43dd None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:48.997351 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86406987-a619-4515-9504-c89c517ba0e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:48.997814 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f63fe4d-4f66-4060-9301-30ccdcaf43dd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:48.998277 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86406987-a619-4515-9504-c89c517ba0e0 None admin] PATH_INFO: `/v3/projects/d13e273646b44ecca3cd10cd867bbd04` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:49.000422 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86406987-a619-4515-9504-c89c517ba0e0 None admin] RBAC: Authorizing `identity:get_project(project_id=d13e273646b44ecca3cd10cd867bbd04)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:49.003461 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86406987-a619-4515-9504-c89c517ba0e0 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:49.009232 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 50/105] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 17 19:09:48 2022] GET /identity/v3/projects/d13e273646b44ecca3cd10cd867bbd04 => generated 298 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:09:49.015347 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86406987-a619-4515-9504-c89c517ba0e0 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:49.024219 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f63fe4d-4f66-4060-9301-30ccdcaf43dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 17 19:09:49.036848 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-3f63fe4d-4f66-4060-9301-30ccdcaf43dd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 17 19:09:49.036848 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:09:49.036848 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:09:49.036848 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:09:49.036848 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:09:49.036848 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:09:49.036848 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:09:49.036848 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:09:49.036848 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:09:49.036848 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:09:49.036848 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:09:49.036848 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:09:49.036848 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 17 19:09:49.036848 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) May 17 19:09:49.036848 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 17 19:09:49.036848 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 17 19:09:49.037710 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 17 19:09:49.037710 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 17 19:09:49.037710 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 17 19:09:49.037710 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 17 19:09:49.037710 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:09:49.037710 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:09:49.037710 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:09:49.037710 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:09:49.037710 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:09:49.037710 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:09:49.037710 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:09:49.037710 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:09:49.037710 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:09:49.037710 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:09:49.037710 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:09:49.037710 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:09:49.038558 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:09:49.038558 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:09:49.038558 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:09:49.038558 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:09:49.038558 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 17 19:09:49.038558 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 17 19:09:49.038558 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 17 19:09:49.038558 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 17 19:09:49.038558 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 17 19:09:49.038558 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 17 19:09:49.038558 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 17 19:09:49.038558 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:09:49.046882 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 56/106] 10.222.0.58 () {56 vars in 1096 bytes} [Tue May 17 19:09:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 17 19:09:49.069418 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f63fe4d-4f66-4060-9301-30ccdcaf43dd None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:49.223587 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86406987-a619-4515-9504-c89c517ba0e0 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:49.235390 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f63fe4d-4f66-4060-9301-30ccdcaf43dd None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:49.262907 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea67236-5d61-4d95-8d1e-a4af4f22eadb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:49.267939 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea67236-5d61-4d95-8d1e-a4af4f22eadb None admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:49.268931 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea67236-5d61-4d95-8d1e-a4af4f22eadb None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:49.269874 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea67236-5d61-4d95-8d1e-a4af4f22eadb None admin] PATH_INFO: `/v3/projects/d13e273646b44ecca3cd10cd867bbd04/users/ed3e18e472cf4c8fb0e0f8e477b961b1/roles/114e6a16e63447d1924707ace5aace25` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:49.273324 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea67236-5d61-4d95-8d1e-a4af4f22eadb None admin] RBAC: Authorizing `identity:create_grant(project_id=d13e273646b44ecca3cd10cd867bbd04, user_id=ed3e18e472cf4c8fb0e0f8e477b961b1, role_id=114e6a16e63447d1924707ace5aace25)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:49.274863 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea67236-5d61-4d95-8d1e-a4af4f22eadb None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:49.277713 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7a8be5-afa5-4d0e-96c8-04dbee125ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:49.278573 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7a8be5-afa5-4d0e-96c8-04dbee125ef3 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:49.278678 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7a8be5-afa5-4d0e-96c8-04dbee125ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:49.278925 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7a8be5-afa5-4d0e-96c8-04dbee125ef3 None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:49.279710 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7a8be5-afa5-4d0e-96c8-04dbee125ef3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:09:49.280117 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7a8be5-afa5-4d0e-96c8-04dbee125ef3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:49.281363 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7a8be5-afa5-4d0e-96c8-04dbee125ef3 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:49.292180 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 57/107] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 17 19:09:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:09:49.293474 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13076003-86a3-4037-9e36-5d4364c2bfff None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:49.293762 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13076003-86a3-4037-9e36-5d4364c2bfff None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:49.294089 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13076003-86a3-4037-9e36-5d4364c2bfff None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:49.295319 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 58/108] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:09:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:09:49.300144 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13076003-86a3-4037-9e36-5d4364c2bfff None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:49.363709 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 51/109] 10.222.0.58 () {58 vars in 1439 bytes} [Tue May 17 19:09:49 2022] PUT /identity/v3/projects/d13e273646b44ecca3cd10cd867bbd04/users/ed3e18e472cf4c8fb0e0f8e477b961b1/roles/114e6a16e63447d1924707ace5aace25 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 17 19:09:49.365380 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5fac7c-312d-401c-8f54-113651d4edec None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:49.365863 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5fac7c-312d-401c-8f54-113651d4edec None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:49.366336 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5fac7c-312d-401c-8f54-113651d4edec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:49.397787 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-9c5fac7c-312d-401c-8f54-113651d4edec None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:09:49.399475 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-9c5fac7c-312d-401c-8f54-113651d4edec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:09:49.416900 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13076003-86a3-4037-9e36-5d4364c2bfff None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:49.481170 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e17ff1d-4696-4b92-8407-639e94f53583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:49.482532 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e17ff1d-4696-4b92-8407-639e94f53583 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:49.484559 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e17ff1d-4696-4b92-8407-639e94f53583 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:49.485045 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e17ff1d-4696-4b92-8407-639e94f53583 None admin] PATH_INFO: `/v3/users/ac2441e330cd41aaa04d125461331f3b` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:49.487505 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e17ff1d-4696-4b92-8407-639e94f53583 None admin] RBAC: Authorizing `identity:get_user(user_id=ac2441e330cd41aaa04d125461331f3b)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:49.490174 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e17ff1d-4696-4b92-8407-639e94f53583 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:49.494616 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 59/110] 10.222.0.58 () {56 vars in 1177 bytes} [Tue May 17 19:09:49 2022] GET /identity/v3/users/ac2441e330cd41aaa04d125461331f3b => generated 247 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:09:49.502740 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e17ff1d-4696-4b92-8407-639e94f53583 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:49.544467 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 52/111] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:09:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 17 19:09:49.590100 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5fac7c-312d-401c-8f54-113651d4edec None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:49.693686 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e17ff1d-4696-4b92-8407-639e94f53583 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:49.731265 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cac9950-a989-491d-99c8-7c01667ceb97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:49.732267 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cac9950-a989-491d-99c8-7c01667ceb97 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:49.732557 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cac9950-a989-491d-99c8-7c01667ceb97 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:49.732835 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cac9950-a989-491d-99c8-7c01667ceb97 None admin] PATH_INFO: `/v3/projects/d13e273646b44ecca3cd10cd867bbd04` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:49.733985 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5fac7c-312d-401c-8f54-113651d4edec None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:49.734158 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cac9950-a989-491d-99c8-7c01667ceb97 None admin] RBAC: Authorizing `identity:get_project(project_id=d13e273646b44ecca3cd10cd867bbd04)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:49.735599 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cac9950-a989-491d-99c8-7c01667ceb97 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:49.737510 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 60/112] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 17 19:09:49 2022] GET /identity/v3/projects/d13e273646b44ecca3cd10cd867bbd04 => generated 298 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:09:49.738561 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5c5a6f-e2de-4368-bc77-a6b5d1cd4329 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:49.738845 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5c5a6f-e2de-4368-bc77-a6b5d1cd4329 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:49.739183 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5c5a6f-e2de-4368-bc77-a6b5d1cd4329 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:49.740615 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 61/113] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:09:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:09:49.744380 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5c5a6f-e2de-4368-bc77-a6b5d1cd4329 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:49.771017 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2e77a6-1351-430a-a8d0-51c30a1bccff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:49.772331 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2e77a6-1351-430a-a8d0-51c30a1bccff None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:49.772823 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2e77a6-1351-430a-a8d0-51c30a1bccff None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:49.773814 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2e77a6-1351-430a-a8d0-51c30a1bccff None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:49.787217 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2e77a6-1351-430a-a8d0-51c30a1bccff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 17 19:09:49.794620 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-7d2e77a6-1351-430a-a8d0-51c30a1bccff None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 17 19:09:49.794620 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:09:49.794620 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:09:49.794620 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:09:49.794620 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:09:49.794620 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:09:49.794620 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:09:49.794620 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:09:49.794620 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:09:49.794620 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:09:49.794620 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:09:49.794620 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:09:49.794620 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 17 19:09:49.794620 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) May 17 19:09:49.794620 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 17 19:09:49.794620 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 17 19:09:49.795455 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 17 19:09:49.795455 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 17 19:09:49.795455 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 17 19:09:49.795455 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 17 19:09:49.795455 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:09:49.795455 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:09:49.795455 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:09:49.795455 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:09:49.795455 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:09:49.795455 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:09:49.795455 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:09:49.795455 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:09:49.795455 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:09:49.795455 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:09:49.795455 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:09:49.795455 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:09:49.796260 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:09:49.796260 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:09:49.796260 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:09:49.796260 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:09:49.796260 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 17 19:09:49.796260 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 17 19:09:49.796260 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 17 19:09:49.796260 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 17 19:09:49.796260 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 17 19:09:49.796260 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 17 19:09:49.796260 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 17 19:09:49.796260 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:09:49.796260 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 53/114] 10.222.0.58 () {56 vars in 1114 bytes} [Tue May 17 19:09:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 17 19:09:49.797666 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67592bcb-4127-4715-bbf4-4c1d1ca391b7 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:49.799280 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67592bcb-4127-4715-bbf4-4c1d1ca391b7 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:49.799280 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67592bcb-4127-4715-bbf4-4c1d1ca391b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:49.815018 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-67592bcb-4127-4715-bbf4-4c1d1ca391b7 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:09:49.816321 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-67592bcb-4127-4715-bbf4-4c1d1ca391b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:09:49.870933 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5c5a6f-e2de-4368-bc77-a6b5d1cd4329 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:49.885714 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 54/115] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:09:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 17 19:09:49.889305 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c55a3d-c658-4a81-abb0-1f25a6710fbe None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:49.889805 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c55a3d-c658-4a81-abb0-1f25a6710fbe None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:49.890294 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c55a3d-c658-4a81-abb0-1f25a6710fbe None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:49.892635 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 55/116] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:09:49 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:09:49.893611 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c55a3d-c658-4a81-abb0-1f25a6710fbe None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:49.910756 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6039b4-2e8e-4010-b95f-662079986369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:49.911677 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6039b4-2e8e-4010-b95f-662079986369 None admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:49.911962 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6039b4-2e8e-4010-b95f-662079986369 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:49.912232 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6039b4-2e8e-4010-b95f-662079986369 None admin] PATH_INFO: `/v3/projects/d13e273646b44ecca3cd10cd867bbd04/users/ac2441e330cd41aaa04d125461331f3b/roles/60af5088793e4379b88821a86552a344` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:49.914741 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6039b4-2e8e-4010-b95f-662079986369 None admin] RBAC: Authorizing `identity:create_grant(project_id=d13e273646b44ecca3cd10cd867bbd04, user_id=ac2441e330cd41aaa04d125461331f3b, role_id=60af5088793e4379b88821a86552a344)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:49.916233 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6039b4-2e8e-4010-b95f-662079986369 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:49.996204 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c55a3d-c658-4a81-abb0-1f25a6710fbe None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:50.021272 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 62/117] 10.222.0.58 () {58 vars in 1439 bytes} [Tue May 17 19:09:49 2022] PUT /identity/v3/projects/d13e273646b44ecca3cd10cd867bbd04/users/ac2441e330cd41aaa04d125461331f3b/roles/60af5088793e4379b88821a86552a344 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:09:50.022662 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f1fdc9-1bac-4418-b6a5-1eaae9d55a9e None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:50.022951 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f1fdc9-1bac-4418-b6a5-1eaae9d55a9e None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:50.023229 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f1fdc9-1bac-4418-b6a5-1eaae9d55a9e None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:50.025233 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 63/118] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:09:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:09:50.026331 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55db60fd-f535-4b94-a459-79f0c82dc6af None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:50.026593 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55db60fd-f535-4b94-a459-79f0c82dc6af None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:50.026847 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55db60fd-f535-4b94-a459-79f0c82dc6af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:50.036852 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee44e16-df68-40db-a6ce-e0c7787350da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:50.037941 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee44e16-df68-40db-a6ce-e0c7787350da None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:50.038537 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee44e16-df68-40db-a6ce-e0c7787350da None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:50.039730 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee44e16-df68-40db-a6ce-e0c7787350da None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:50.040876 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee44e16-df68-40db-a6ce-e0c7787350da None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:09:50.041518 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee44e16-df68-40db-a6ce-e0c7787350da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:50.043331 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee44e16-df68-40db-a6ce-e0c7787350da None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:50.048032 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-55db60fd-f535-4b94-a459-79f0c82dc6af None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:09:50.049387 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-55db60fd-f535-4b94-a459-79f0c82dc6af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:09:50.054822 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 56/119] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 17 19:09:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:09:50.056066 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee44e16-df68-40db-a6ce-e0c7787350da None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:50.125774 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 64/120] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:09:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 17 19:09:50.133260 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9957b42-0e1d-4ae0-91fa-92aa2389165c None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:50.133596 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9957b42-0e1d-4ae0-91fa-92aa2389165c None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:50.134429 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9957b42-0e1d-4ae0-91fa-92aa2389165c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:50.163319 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-e9957b42-0e1d-4ae0-91fa-92aa2389165c None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:09:50.164625 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-e9957b42-0e1d-4ae0-91fa-92aa2389165c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:09:50.180540 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee44e16-df68-40db-a6ce-e0c7787350da None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:50.217778 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5e05d7-82bf-4f9d-9172-485e7ca83d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:50.218950 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5e05d7-82bf-4f9d-9172-485e7ca83d3d None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:50.219427 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5e05d7-82bf-4f9d-9172-485e7ca83d3d None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:50.219918 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5e05d7-82bf-4f9d-9172-485e7ca83d3d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:50.231007 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5e05d7-82bf-4f9d-9172-485e7ca83d3d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 17 19:09:50.235381 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 65/121] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:09:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 17 19:09:50.236347 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9957b42-0e1d-4ae0-91fa-92aa2389165c None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:50.239950 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-3a5e05d7-82bf-4f9d-9172-485e7ca83d3d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 17 19:09:50.239950 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:09:50.239950 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:09:50.239950 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:09:50.239950 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:09:50.239950 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:09:50.239950 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:09:50.239950 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:09:50.239950 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:09:50.239950 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:09:50.239950 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:09:50.239950 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:09:50.239950 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 17 19:09:50.239950 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) May 17 19:09:50.239950 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 17 19:09:50.239950 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 17 19:09:50.240807 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 17 19:09:50.240807 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 17 19:09:50.240807 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 17 19:09:50.240807 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 17 19:09:50.240807 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:09:50.240807 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:09:50.240807 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:09:50.240807 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:09:50.240807 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:09:50.240807 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:09:50.240807 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:09:50.240807 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:09:50.240807 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:09:50.240807 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:09:50.240807 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:09:50.240807 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:09:50.241559 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:09:50.241559 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:09:50.241559 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:09:50.241559 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:09:50.241559 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 17 19:09:50.241559 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 17 19:09:50.241559 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 17 19:09:50.241559 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 17 19:09:50.241559 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 17 19:09:50.241559 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 17 19:09:50.241559 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 17 19:09:50.241559 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:09:50.244302 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 57/122] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 17 19:09:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:09:50.245292 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5e05d7-82bf-4f9d-9172-485e7ca83d3d None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:50.337402 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9957b42-0e1d-4ae0-91fa-92aa2389165c None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:50.351935 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5e05d7-82bf-4f9d-9172-485e7ca83d3d None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:50.371403 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1299e347-9b5e-4c37-a425-be9da909484c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:50.372311 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1299e347-9b5e-4c37-a425-be9da909484c None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:50.372595 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1299e347-9b5e-4c37-a425-be9da909484c None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:50.372874 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1299e347-9b5e-4c37-a425-be9da909484c None admin] PATH_INFO: `/v3/users/ed3e18e472cf4c8fb0e0f8e477b961b1` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:50.374161 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1299e347-9b5e-4c37-a425-be9da909484c None admin] RBAC: Authorizing `identity:get_user(user_id=ed3e18e472cf4c8fb0e0f8e477b961b1)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:50.375615 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1299e347-9b5e-4c37-a425-be9da909484c None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:50.377245 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 66/123] 10.222.0.58 () {56 vars in 1177 bytes} [Tue May 17 19:09:50 2022] GET /identity/v3/users/ed3e18e472cf4c8fb0e0f8e477b961b1 => generated 275 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:09:50.378064 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1299e347-9b5e-4c37-a425-be9da909484c None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:50.389235 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cce0b0-de31-432f-86b6-1681433ba0a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:50.390322 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cce0b0-de31-432f-86b6-1681433ba0a9 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:50.390815 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cce0b0-de31-432f-86b6-1681433ba0a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:50.391287 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cce0b0-de31-432f-86b6-1681433ba0a9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:50.392681 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cce0b0-de31-432f-86b6-1681433ba0a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:50.394358 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cce0b0-de31-432f-86b6-1681433ba0a9 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:50.396565 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 58/124] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:09:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:09:50.402236 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cce0b0-de31-432f-86b6-1681433ba0a9 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:50.477651 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1299e347-9b5e-4c37-a425-be9da909484c None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:50.502905 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cce0b0-de31-432f-86b6-1681433ba0a9 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:50.512666 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a92ebb-c3c0-4e8d-8ca3-a7da82bca4a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:50.513576 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a92ebb-c3c0-4e8d-8ca3-a7da82bca4a2 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:50.513576 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a92ebb-c3c0-4e8d-8ca3-a7da82bca4a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:50.513868 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a92ebb-c3c0-4e8d-8ca3-a7da82bca4a2 None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:50.514620 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a92ebb-c3c0-4e8d-8ca3-a7da82bca4a2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:09:50.514923 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a92ebb-c3c0-4e8d-8ca3-a7da82bca4a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:50.516407 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a92ebb-c3c0-4e8d-8ca3-a7da82bca4a2 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:50.529314 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 67/125] 10.222.0.58 () {56 vars in 1101 bytes} [Tue May 17 19:09:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:09:50.529314 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a92ebb-c3c0-4e8d-8ca3-a7da82bca4a2 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:50.548313 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e812ae91-022b-4ff8-8413-05c9c2fe888d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:50.549277 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e812ae91-022b-4ff8-8413-05c9c2fe888d None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:50.549568 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e812ae91-022b-4ff8-8413-05c9c2fe888d None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:50.549848 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e812ae91-022b-4ff8-8413-05c9c2fe888d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:50.564113 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e812ae91-022b-4ff8-8413-05c9c2fe888d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 17 19:09:50.575104 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-e812ae91-022b-4ff8-8413-05c9c2fe888d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 17 19:09:50.575104 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:09:50.575104 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:09:50.575104 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:09:50.575104 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:09:50.575104 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:09:50.575104 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:09:50.575104 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:09:50.575104 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:09:50.575104 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:09:50.575104 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:09:50.575104 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:09:50.575104 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 17 19:09:50.575104 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) May 17 19:09:50.575104 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 17 19:09:50.575104 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 17 19:09:50.576016 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 17 19:09:50.576016 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 17 19:09:50.576016 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 17 19:09:50.576016 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 17 19:09:50.576016 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:09:50.576016 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:09:50.576016 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:09:50.576016 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:09:50.576016 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:09:50.576016 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:09:50.576016 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:09:50.576016 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:09:50.576016 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:09:50.576016 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:09:50.576016 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:09:50.576016 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:09:50.576761 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:09:50.576761 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:09:50.576761 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:09:50.576761 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:09:50.576761 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 17 19:09:50.576761 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 17 19:09:50.576761 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 17 19:09:50.576761 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 17 19:09:50.576761 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 17 19:09:50.576761 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 17 19:09:50.576761 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 17 19:09:50.576761 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:09:50.576761 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 59/126] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 17 19:09:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:09:50.577920 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e812ae91-022b-4ff8-8413-05c9c2fe888d None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:50.651259 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a92ebb-c3c0-4e8d-8ca3-a7da82bca4a2 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:50.686877 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abccb32b-bd5d-4e13-965a-a70dcb13ef59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:50.687087 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e812ae91-022b-4ff8-8413-05c9c2fe888d None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:50.687759 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abccb32b-bd5d-4e13-965a-a70dcb13ef59 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:50.688038 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abccb32b-bd5d-4e13-965a-a70dcb13ef59 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:50.688311 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abccb32b-bd5d-4e13-965a-a70dcb13ef59 None admin] PATH_INFO: `/v3/projects/d13e273646b44ecca3cd10cd867bbd04` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:50.689588 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abccb32b-bd5d-4e13-965a-a70dcb13ef59 None admin] RBAC: Authorizing `identity:get_project(project_id=d13e273646b44ecca3cd10cd867bbd04)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:50.691096 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abccb32b-bd5d-4e13-965a-a70dcb13ef59 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:50.693155 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 68/127] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 17 19:09:50 2022] GET /identity/v3/projects/d13e273646b44ecca3cd10cd867bbd04 => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:09:50.693935 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abccb32b-bd5d-4e13-965a-a70dcb13ef59 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:50.725855 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10e8eff-094c-4942-804e-7ba4865fe98d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:50.726547 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10e8eff-094c-4942-804e-7ba4865fe98d None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:50.726834 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10e8eff-094c-4942-804e-7ba4865fe98d None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:50.727112 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10e8eff-094c-4942-804e-7ba4865fe98d None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:50.727890 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10e8eff-094c-4942-804e-7ba4865fe98d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:50.729385 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10e8eff-094c-4942-804e-7ba4865fe98d None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:50.793314 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abccb32b-bd5d-4e13-965a-a70dcb13ef59 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:50.826034 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba2ed94-0b8e-4e57-93fa-365e2a5954ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:50.826906 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba2ed94-0b8e-4e57-93fa-365e2a5954ae None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:50.827188 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba2ed94-0b8e-4e57-93fa-365e2a5954ae None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:50.827460 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba2ed94-0b8e-4e57-93fa-365e2a5954ae None admin] PATH_INFO: `/v3/users/ce8e7192e8fc44da9892e1a896b57287` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:50.843889 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba2ed94-0b8e-4e57-93fa-365e2a5954ae None admin] RBAC: Authorizing `identity:get_user(user_id=ce8e7192e8fc44da9892e1a896b57287)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:50.845577 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba2ed94-0b8e-4e57-93fa-365e2a5954ae None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:50.855306 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 60/128] 10.222.0.58 () {60 vars in 1141 bytes} [Tue May 17 19:09:50 2022] POST /identity/v3/projects => generated 302 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:09:50.855456 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 69/129] 10.222.0.58 () {56 vars in 1177 bytes} [Tue May 17 19:09:50 2022] GET /identity/v3/users/ce8e7192e8fc44da9892e1a896b57287 => generated 289 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:09:50.855568 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10e8eff-094c-4942-804e-7ba4865fe98d None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:50.855648 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba2ed94-0b8e-4e57-93fa-365e2a5954ae None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:50.964750 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10e8eff-094c-4942-804e-7ba4865fe98d None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:50.967372 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba2ed94-0b8e-4e57-93fa-365e2a5954ae None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:51.001435 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a24e77c-1c08-4423-a466-fbe69d937946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:51.002118 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a24e77c-1c08-4423-a466-fbe69d937946 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:51.002409 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a24e77c-1c08-4423-a466-fbe69d937946 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:51.002669 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a24e77c-1c08-4423-a466-fbe69d937946 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:51.003429 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a24e77c-1c08-4423-a466-fbe69d937946 None admin] RBAC: Adding query filter params (name=member) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:09:51.003850 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a24e77c-1c08-4423-a466-fbe69d937946 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:51.005331 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a24e77c-1c08-4423-a466-fbe69d937946 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:51.005768 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89e0562-f11c-4c84-bcb8-c5e3d3a41a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:51.006847 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89e0562-f11c-4c84-bcb8-c5e3d3a41a16 None admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:51.007301 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89e0562-f11c-4c84-bcb8-c5e3d3a41a16 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:51.007823 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89e0562-f11c-4c84-bcb8-c5e3d3a41a16 None admin] PATH_INFO: `/v3/projects/d13e273646b44ecca3cd10cd867bbd04/users/ed3e18e472cf4c8fb0e0f8e477b961b1/roles/1055a3235b0143e58f1090c1e154e5c1` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:51.010276 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89e0562-f11c-4c84-bcb8-c5e3d3a41a16 None admin] RBAC: Authorizing `identity:create_grant(project_id=d13e273646b44ecca3cd10cd867bbd04, user_id=ed3e18e472cf4c8fb0e0f8e477b961b1, role_id=1055a3235b0143e58f1090c1e154e5c1)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:51.012071 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89e0562-f11c-4c84-bcb8-c5e3d3a41a16 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:51.014772 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 61/130] 10.222.0.58 () {56 vars in 1101 bytes} [Tue May 17 19:09:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:09:51.019792 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a24e77c-1c08-4423-a466-fbe69d937946 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:51.129798 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 70/131] 10.222.0.58 () {58 vars in 1439 bytes} [Tue May 17 19:09:50 2022] PUT /identity/v3/projects/d13e273646b44ecca3cd10cd867bbd04/users/ed3e18e472cf4c8fb0e0f8e477b961b1/roles/1055a3235b0143e58f1090c1e154e5c1 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:09:51.130721 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89e0562-f11c-4c84-bcb8-c5e3d3a41a16 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:51.140948 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a24e77c-1c08-4423-a466-fbe69d937946 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:51.184126 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4ea124-ccd7-4df9-be2c-552ea06a3560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:51.185710 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4ea124-ccd7-4df9-be2c-552ea06a3560 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:51.185710 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4ea124-ccd7-4df9-be2c-552ea06a3560 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:51.185710 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4ea124-ccd7-4df9-be2c-552ea06a3560 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:51.269016 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4ea124-ccd7-4df9-be2c-552ea06a3560 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:51.270444 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4ea124-ccd7-4df9-be2c-552ea06a3560 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:51.274733 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89e0562-f11c-4c84-bcb8-c5e3d3a41a16 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:51.320835 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdf43ed-128c-468e-b2bb-f2b54e8a4ab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:51.322794 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdf43ed-128c-468e-b2bb-f2b54e8a4ab2 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:51.323302 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdf43ed-128c-468e-b2bb-f2b54e8a4ab2 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:51.323796 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdf43ed-128c-468e-b2bb-f2b54e8a4ab2 None admin] PATH_INFO: `/v3/users/ed3e18e472cf4c8fb0e0f8e477b961b1` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:51.325823 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdf43ed-128c-468e-b2bb-f2b54e8a4ab2 None admin] RBAC: Authorizing `identity:get_user(user_id=ed3e18e472cf4c8fb0e0f8e477b961b1)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:51.328127 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdf43ed-128c-468e-b2bb-f2b54e8a4ab2 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:51.330528 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 71/132] 10.222.0.58 () {56 vars in 1177 bytes} [Tue May 17 19:09:51 2022] GET /identity/v3/users/ed3e18e472cf4c8fb0e0f8e477b961b1 => generated 275 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:09:51.340902 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdf43ed-128c-468e-b2bb-f2b54e8a4ab2 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:51.434494 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 62/133] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 17 19:09:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 17 19:09:51.441618 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4ea124-ccd7-4df9-be2c-552ea06a3560 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:51.543593 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdf43ed-128c-468e-b2bb-f2b54e8a4ab2 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:51.583874 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6547dccd-23db-4516-a718-7ebb6665224f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:51.584746 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6547dccd-23db-4516-a718-7ebb6665224f None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:51.585033 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6547dccd-23db-4516-a718-7ebb6665224f None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:51.585311 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6547dccd-23db-4516-a718-7ebb6665224f None admin] PATH_INFO: `/v3/projects/e2de3fd8bde6430aa6a1e97114f6aa87` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:51.586548 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6547dccd-23db-4516-a718-7ebb6665224f None admin] RBAC: Authorizing `identity:get_project(project_id=e2de3fd8bde6430aa6a1e97114f6aa87)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:51.588958 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6547dccd-23db-4516-a718-7ebb6665224f None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:51.591192 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 72/134] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 17 19:09:51 2022] GET /identity/v3/projects/e2de3fd8bde6430aa6a1e97114f6aa87 => generated 312 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:09:51.596873 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6547dccd-23db-4516-a718-7ebb6665224f None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:51.607828 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4ea124-ccd7-4df9-be2c-552ea06a3560 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:51.653767 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3686bb96-c7b6-4d56-a5d2-61b7f0495ac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:51.654990 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3686bb96-c7b6-4d56-a5d2-61b7f0495ac8 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:51.655515 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3686bb96-c7b6-4d56-a5d2-61b7f0495ac8 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:51.656156 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3686bb96-c7b6-4d56-a5d2-61b7f0495ac8 None admin] PATH_INFO: `/v3/projects/d13e273646b44ecca3cd10cd867bbd04` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:51.657770 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3686bb96-c7b6-4d56-a5d2-61b7f0495ac8 None admin] RBAC: Authorizing `identity:get_project(project_id=d13e273646b44ecca3cd10cd867bbd04)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:51.660167 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3686bb96-c7b6-4d56-a5d2-61b7f0495ac8 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:51.662115 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 63/135] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 17 19:09:51 2022] GET /identity/v3/projects/d13e273646b44ecca3cd10cd867bbd04 => generated 298 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:09:51.668465 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3686bb96-c7b6-4d56-a5d2-61b7f0495ac8 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:51.731461 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6547dccd-23db-4516-a718-7ebb6665224f None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:51.766820 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0a2426-099c-47c6-bb69-f12c67def022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:51.768087 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0a2426-099c-47c6-bb69-f12c67def022 None admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:51.768547 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0a2426-099c-47c6-bb69-f12c67def022 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:51.769120 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0a2426-099c-47c6-bb69-f12c67def022 None admin] PATH_INFO: `/v3/projects/e2de3fd8bde6430aa6a1e97114f6aa87/users/ed3e18e472cf4c8fb0e0f8e477b961b1/roles/114e6a16e63447d1924707ace5aace25` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:51.771315 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0a2426-099c-47c6-bb69-f12c67def022 None admin] RBAC: Authorizing `identity:create_grant(project_id=e2de3fd8bde6430aa6a1e97114f6aa87, user_id=ed3e18e472cf4c8fb0e0f8e477b961b1, role_id=114e6a16e63447d1924707ace5aace25)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:51.772988 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0a2426-099c-47c6-bb69-f12c67def022 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:51.840933 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 73/136] 10.222.0.58 () {58 vars in 1439 bytes} [Tue May 17 19:09:51 2022] PUT /identity/v3/projects/e2de3fd8bde6430aa6a1e97114f6aa87/users/ed3e18e472cf4c8fb0e0f8e477b961b1/roles/114e6a16e63447d1924707ace5aace25 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:09:51.871256 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3686bb96-c7b6-4d56-a5d2-61b7f0495ac8 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:51.932583 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a84cdc-e8f4-44d8-b978-68d77ac39391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:51.933265 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a84cdc-e8f4-44d8-b978-68d77ac39391 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:51.933553 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a84cdc-e8f4-44d8-b978-68d77ac39391 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:51.933829 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a84cdc-e8f4-44d8-b978-68d77ac39391 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:51.934685 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a84cdc-e8f4-44d8-b978-68d77ac39391 None admin] RBAC: Adding query filter params (role.id=8e3500b266214ae2be189641a1b4dbf1, scope.project.id=d13e273646b44ecca3cd10cd867bbd04, user.id=ce8e7192e8fc44da9892e1a896b57287) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:09:51.934997 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a84cdc-e8f4-44d8-b978-68d77ac39391 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:51.936542 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a84cdc-e8f4-44d8-b978-68d77ac39391 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:51.954489 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 64/137] 10.222.0.58 () {56 vars in 1374 bytes} [Tue May 17 19:09:51 2022] GET /identity/v3/role_assignments?user.id=ce8e7192e8fc44da9892e1a896b57287&scope.project.id=d13e273646b44ecca3cd10cd867bbd04&role.id=8e3500b266214ae2be189641a1b4dbf1 => generated 259 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:09:52.251660 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ec1259-7d78-418a-9175-96f66f50db69 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:52.252196 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ec1259-7d78-418a-9175-96f66f50db69 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:52.252196 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ec1259-7d78-418a-9175-96f66f50db69 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:52.253800 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 74/138] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:09:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 17 19:09:52.259832 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3748ff6-3142-4c05-84e7-1c4d4c5fc7c6 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:52.260233 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3748ff6-3142-4c05-84e7-1c4d4c5fc7c6 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:52.260423 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3748ff6-3142-4c05-84e7-1c4d4c5fc7c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:52.284500 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-c3748ff6-3142-4c05-84e7-1c4d4c5fc7c6 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:09:52.285691 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c3748ff6-3142-4c05-84e7-1c4d4c5fc7c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:09:52.365247 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 65/139] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:09:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 17 19:09:52.400033 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ec1259-7d78-418a-9175-96f66f50db69 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:52.525529 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ec1259-7d78-418a-9175-96f66f50db69 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:52.567903 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c09d6f-7208-4ec1-a588-eada81644d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:52.568700 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c09d6f-7208-4ec1-a588-eada81644d2a None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:52.568981 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c09d6f-7208-4ec1-a588-eada81644d2a None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:52.569254 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c09d6f-7208-4ec1-a588-eada81644d2a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:52.588415 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c09d6f-7208-4ec1-a588-eada81644d2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 17 19:09:52.596098 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-03c09d6f-7208-4ec1-a588-eada81644d2a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 17 19:09:52.596098 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:09:52.596098 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:09:52.596098 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:09:52.596098 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:09:52.596098 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:09:52.596098 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:09:52.596098 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:09:52.596098 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:09:52.596098 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:09:52.596098 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:09:52.596098 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:09:52.596098 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 17 19:09:52.596098 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) May 17 19:09:52.596098 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 17 19:09:52.596098 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 17 19:09:52.597189 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 17 19:09:52.597189 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 17 19:09:52.597189 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 17 19:09:52.597189 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 17 19:09:52.597189 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:09:52.597189 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:09:52.597189 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:09:52.597189 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:09:52.597189 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:09:52.597189 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:09:52.597189 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:09:52.597189 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:09:52.597189 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:09:52.597189 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:09:52.597189 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:09:52.597189 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:09:52.597961 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:09:52.597961 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:09:52.597961 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:09:52.597961 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:09:52.597961 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 17 19:09:52.597961 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 17 19:09:52.597961 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 17 19:09:52.597961 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 17 19:09:52.597961 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 17 19:09:52.597961 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 17 19:09:52.597961 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 17 19:09:52.597961 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:09:52.597961 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 75/140] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 17 19:09:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:09:52.605804 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3748ff6-3142-4c05-84e7-1c4d4c5fc7c6 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:52.729905 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3748ff6-3142-4c05-84e7-1c4d4c5fc7c6 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:52.767198 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbd7d5c-8587-46e0-a8f5-e63d48bc21b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:52.767952 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbd7d5c-8587-46e0-a8f5-e63d48bc21b7 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:52.768239 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbd7d5c-8587-46e0-a8f5-e63d48bc21b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:52.768646 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbd7d5c-8587-46e0-a8f5-e63d48bc21b7 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:52.769406 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbd7d5c-8587-46e0-a8f5-e63d48bc21b7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:09:52.769709 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbd7d5c-8587-46e0-a8f5-e63d48bc21b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:52.771093 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbd7d5c-8587-46e0-a8f5-e63d48bc21b7 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:52.780160 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 66/141] 10.222.0.58 () {56 vars in 1101 bytes} [Tue May 17 19:09:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:09:52.788608 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c09d6f-7208-4ec1-a588-eada81644d2a None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:52.919116 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c09d6f-7208-4ec1-a588-eada81644d2a None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:52.970011 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486a0285-455d-4640-b3b5-2ab339fdad6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:52.970889 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486a0285-455d-4640-b3b5-2ab339fdad6c None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:52.971178 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486a0285-455d-4640-b3b5-2ab339fdad6c None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:52.971454 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486a0285-455d-4640-b3b5-2ab339fdad6c None admin] PATH_INFO: `/v3/users/ed3e18e472cf4c8fb0e0f8e477b961b1` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:52.972953 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486a0285-455d-4640-b3b5-2ab339fdad6c None admin] RBAC: Authorizing `identity:get_user(user_id=ed3e18e472cf4c8fb0e0f8e477b961b1)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:52.974389 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486a0285-455d-4640-b3b5-2ab339fdad6c None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:52.976044 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 76/142] 10.222.0.58 () {56 vars in 1177 bytes} [Tue May 17 19:09:52 2022] GET /identity/v3/users/ed3e18e472cf4c8fb0e0f8e477b961b1 => generated 275 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:09:52.982093 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbd7d5c-8587-46e0-a8f5-e63d48bc21b7 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:53.111698 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea448f0-8199-46aa-b2d6-4455913046ab None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:53.111983 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea448f0-8199-46aa-b2d6-4455913046ab None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:53.112190 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea448f0-8199-46aa-b2d6-4455913046ab None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:53.113800 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 77/143] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:09:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:09:53.118798 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9b0866-173a-4cb1-9e5d-36945126e4d3 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:53.119104 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9b0866-173a-4cb1-9e5d-36945126e4d3 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:53.119370 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9b0866-173a-4cb1-9e5d-36945126e4d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:53.128455 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbd7d5c-8587-46e0-a8f5-e63d48bc21b7 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:53.152501 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-5c9b0866-173a-4cb1-9e5d-36945126e4d3 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:09:53.153723 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-5c9b0866-173a-4cb1-9e5d-36945126e4d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:09:53.164424 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e1c7df-6973-42a3-9451-0214a823e806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:53.165712 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e1c7df-6973-42a3-9451-0214a823e806 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:53.166136 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e1c7df-6973-42a3-9451-0214a823e806 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:53.166552 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e1c7df-6973-42a3-9451-0214a823e806 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:53.296094 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e1c7df-6973-42a3-9451-0214a823e806 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:53.298919 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e1c7df-6973-42a3-9451-0214a823e806 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:53.316060 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 78/144] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:09:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 17 19:09:53.393188 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9b0866-173a-4cb1-9e5d-36945126e4d3 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:53.441565 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 67/145] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 17 19:09:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 17 19:09:53.447830 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e1c7df-6973-42a3-9451-0214a823e806 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:53.505249 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9b0866-173a-4cb1-9e5d-36945126e4d3 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:53.548826 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c23386a-490f-4f7a-97b4-74ec534b065c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:53.549627 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c23386a-490f-4f7a-97b4-74ec534b065c None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:53.549910 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c23386a-490f-4f7a-97b4-74ec534b065c None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:53.550184 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c23386a-490f-4f7a-97b4-74ec534b065c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:53.557329 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e1c7df-6973-42a3-9451-0214a823e806 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:53.568264 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c23386a-490f-4f7a-97b4-74ec534b065c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 17 19:09:53.584517 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-8c23386a-490f-4f7a-97b4-74ec534b065c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 17 19:09:53.584517 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:09:53.584517 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:09:53.584517 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:09:53.584517 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:09:53.584517 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:09:53.584517 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:09:53.584517 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:09:53.584517 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:09:53.584517 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:09:53.584517 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:09:53.584517 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:09:53.584517 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 17 19:09:53.584517 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) May 17 19:09:53.584517 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 17 19:09:53.584517 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 17 19:09:53.585390 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 17 19:09:53.585390 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 17 19:09:53.585390 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 17 19:09:53.585390 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 17 19:09:53.585390 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:09:53.585390 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:09:53.585390 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:09:53.585390 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:09:53.585390 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:09:53.585390 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:09:53.585390 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:09:53.585390 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:09:53.585390 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:09:53.585390 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:09:53.585390 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:09:53.585390 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:09:53.586597 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:09:53.586597 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:09:53.586597 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:09:53.586597 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:09:53.586597 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 17 19:09:53.586597 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 17 19:09:53.586597 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 17 19:09:53.586597 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 17 19:09:53.586597 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 17 19:09:53.586597 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 17 19:09:53.586597 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 17 19:09:53.586597 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:09:53.586597 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 79/146] 10.222.0.58 () {56 vars in 1096 bytes} [Tue May 17 19:09:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 17 19:09:53.591612 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c23386a-490f-4f7a-97b4-74ec534b065c None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:53.604204 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a54dfbf-da6b-4447-91bb-5850cc9c36f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:53.605084 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a54dfbf-da6b-4447-91bb-5850cc9c36f3 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:53.605363 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a54dfbf-da6b-4447-91bb-5850cc9c36f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:53.605692 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a54dfbf-da6b-4447-91bb-5850cc9c36f3 None admin] PATH_INFO: `/v3/projects/d13e273646b44ecca3cd10cd867bbd04` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:53.606981 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a54dfbf-da6b-4447-91bb-5850cc9c36f3 None admin] RBAC: Authorizing `identity:get_project(project_id=d13e273646b44ecca3cd10cd867bbd04)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:53.608669 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a54dfbf-da6b-4447-91bb-5850cc9c36f3 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:53.610375 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 68/147] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 17 19:09:53 2022] GET /identity/v3/projects/d13e273646b44ecca3cd10cd867bbd04 => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:09:53.617712 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a54dfbf-da6b-4447-91bb-5850cc9c36f3 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:53.704247 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c23386a-490f-4f7a-97b4-74ec534b065c None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:53.748741 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f097e714-c3e5-44f5-a509-7397c08d29ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:53.749097 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a54dfbf-da6b-4447-91bb-5850cc9c36f3 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:53.749418 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f097e714-c3e5-44f5-a509-7397c08d29ca None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:53.749706 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f097e714-c3e5-44f5-a509-7397c08d29ca None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:53.749988 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f097e714-c3e5-44f5-a509-7397c08d29ca None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:53.750741 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f097e714-c3e5-44f5-a509-7397c08d29ca None admin] RBAC: Adding query filter params (name=admin) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:09:53.751048 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f097e714-c3e5-44f5-a509-7397c08d29ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:53.752464 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f097e714-c3e5-44f5-a509-7397c08d29ca None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:53.766697 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 80/148] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 17 19:09:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:09:53.772197 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f097e714-c3e5-44f5-a509-7397c08d29ca None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:53.796171 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ab828e-22b1-49aa-84b7-6dbb861f6cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:53.796831 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ab828e-22b1-49aa-84b7-6dbb861f6cff None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:53.797111 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ab828e-22b1-49aa-84b7-6dbb861f6cff None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:53.797381 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ab828e-22b1-49aa-84b7-6dbb861f6cff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:53.798208 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ab828e-22b1-49aa-84b7-6dbb861f6cff None admin] RBAC: Adding query filter params (role.id=114e6a16e63447d1924707ace5aace25, scope.project.id=d13e273646b44ecca3cd10cd867bbd04, user.id=ed3e18e472cf4c8fb0e0f8e477b961b1) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:09:53.798517 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ab828e-22b1-49aa-84b7-6dbb861f6cff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:53.800010 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ab828e-22b1-49aa-84b7-6dbb861f6cff None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:53.806853 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 69/149] 10.222.0.58 () {56 vars in 1374 bytes} [Tue May 17 19:09:53 2022] GET /identity/v3/role_assignments?user.id=ed3e18e472cf4c8fb0e0f8e477b961b1&scope.project.id=d13e273646b44ecca3cd10cd867bbd04&role.id=114e6a16e63447d1924707ace5aace25 => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:09:53.887200 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f097e714-c3e5-44f5-a509-7397c08d29ca None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:53.927008 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbd3e9b-30e7-40c7-80d7-f7b1690b44cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:53.927947 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbd3e9b-30e7-40c7-80d7-f7b1690b44cc None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:53.928197 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbd3e9b-30e7-40c7-80d7-f7b1690b44cc None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:53.928467 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbd3e9b-30e7-40c7-80d7-f7b1690b44cc None admin] PATH_INFO: `/v3/users/ac2441e330cd41aaa04d125461331f3b` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:53.932279 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbd3e9b-30e7-40c7-80d7-f7b1690b44cc None admin] RBAC: Authorizing `identity:get_user(user_id=ac2441e330cd41aaa04d125461331f3b)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:53.933771 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbd3e9b-30e7-40c7-80d7-f7b1690b44cc None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:53.935462 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 81/150] 10.222.0.58 () {56 vars in 1177 bytes} [Tue May 17 19:09:53 2022] GET /identity/v3/users/ac2441e330cd41aaa04d125461331f3b => generated 247 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:09:53.941089 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ab828e-22b1-49aa-84b7-6dbb861f6cff None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:54.077996 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ab828e-22b1-49aa-84b7-6dbb861f6cff None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:54.095732 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e03fe04-9d93-4770-ad5f-ab4fd166971b None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:54.096067 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e03fe04-9d93-4770-ad5f-ab4fd166971b None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:54.096277 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e03fe04-9d93-4770-ad5f-ab4fd166971b None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:54.097918 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 82/151] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:09:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 17 19:09:54.105027 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27d5099-a5a8-4d40-85de-059a14b4dcc3 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:54.105334 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27d5099-a5a8-4d40-85de-059a14b4dcc3 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:54.105575 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27d5099-a5a8-4d40-85de-059a14b4dcc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:54.119255 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431b128d-ae4b-4191-822d-2c826c202831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:54.119947 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431b128d-ae4b-4191-822d-2c826c202831 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:54.120230 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431b128d-ae4b-4191-822d-2c826c202831 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:54.120506 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431b128d-ae4b-4191-822d-2c826c202831 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:54.149092 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-b27d5099-a5a8-4d40-85de-059a14b4dcc3 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:09:54.150294 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-b27d5099-a5a8-4d40-85de-059a14b4dcc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:09:54.238106 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431b128d-ae4b-4191-822d-2c826c202831 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:54.239627 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431b128d-ae4b-4191-822d-2c826c202831 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:54.275209 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 83/152] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:09:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 17 19:09:54.366727 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 70/153] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 17 19:09:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 17 19:09:54.369619 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27d5099-a5a8-4d40-85de-059a14b4dcc3 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:54.380693 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431b128d-ae4b-4191-822d-2c826c202831 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:54.485171 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27d5099-a5a8-4d40-85de-059a14b4dcc3 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:54.530946 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78676df1-b97f-4e21-a0e7-73fd60e8502b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:54.531774 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78676df1-b97f-4e21-a0e7-73fd60e8502b None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:54.532051 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78676df1-b97f-4e21-a0e7-73fd60e8502b None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:54.532326 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78676df1-b97f-4e21-a0e7-73fd60e8502b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:54.536180 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78676df1-b97f-4e21-a0e7-73fd60e8502b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:54.537652 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78676df1-b97f-4e21-a0e7-73fd60e8502b None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:54.539888 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 84/154] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:09:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:09:54.545145 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78676df1-b97f-4e21-a0e7-73fd60e8502b None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:54.559141 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431b128d-ae4b-4191-822d-2c826c202831 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:54.622746 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e916bf-5b59-4b5f-8173-5f5bb904e152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:54.622746 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e916bf-5b59-4b5f-8173-5f5bb904e152 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:54.622746 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e916bf-5b59-4b5f-8173-5f5bb904e152 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:54.622746 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e916bf-5b59-4b5f-8173-5f5bb904e152 None admin] PATH_INFO: `/v3/projects/d13e273646b44ecca3cd10cd867bbd04` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:54.624391 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e916bf-5b59-4b5f-8173-5f5bb904e152 None admin] RBAC: Authorizing `identity:get_project(project_id=d13e273646b44ecca3cd10cd867bbd04)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:54.625893 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e916bf-5b59-4b5f-8173-5f5bb904e152 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:54.627903 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 71/155] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 17 19:09:54 2022] GET /identity/v3/projects/d13e273646b44ecca3cd10cd867bbd04 => generated 298 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:09:54.634435 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e916bf-5b59-4b5f-8173-5f5bb904e152 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:54.728851 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78676df1-b97f-4e21-a0e7-73fd60e8502b None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:54.780733 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e916bf-5b59-4b5f-8173-5f5bb904e152 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:54.799461 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360fd264-3e0d-4ee5-8435-5ed604e6d427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:54.800508 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360fd264-3e0d-4ee5-8435-5ed604e6d427 None admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:54.800998 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360fd264-3e0d-4ee5-8435-5ed604e6d427 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:54.801464 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360fd264-3e0d-4ee5-8435-5ed604e6d427 None admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:54.802415 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360fd264-3e0d-4ee5-8435-5ed604e6d427 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:54.804374 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360fd264-3e0d-4ee5-8435-5ed604e6d427 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:54.819096 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530d4896-2408-469f-af01-c6847538cf72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:54.819783 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530d4896-2408-469f-af01-c6847538cf72 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:54.820105 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530d4896-2408-469f-af01-c6847538cf72 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:54.820386 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530d4896-2408-469f-af01-c6847538cf72 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:54.821565 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530d4896-2408-469f-af01-c6847538cf72 None admin] RBAC: Adding query filter params (role.id=60af5088793e4379b88821a86552a344, scope.project.id=d13e273646b44ecca3cd10cd867bbd04, user.id=ac2441e330cd41aaa04d125461331f3b) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:09:54.821881 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530d4896-2408-469f-af01-c6847538cf72 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:54.823473 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530d4896-2408-469f-af01-c6847538cf72 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:54.829474 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 72/156] 10.222.0.58 () {56 vars in 1374 bytes} [Tue May 17 19:09:54 2022] GET /identity/v3/role_assignments?user.id=ac2441e330cd41aaa04d125461331f3b&scope.project.id=d13e273646b44ecca3cd10cd867bbd04&role.id=60af5088793e4379b88821a86552a344 => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:09:54.830495 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd79f534-e36c-4455-91cd-b382cf8f28d4 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:54.830785 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd79f534-e36c-4455-91cd-b382cf8f28d4 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:54.831059 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd79f534-e36c-4455-91cd-b382cf8f28d4 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:54.839549 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 73/157] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:09:54 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:09:54.839549 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138da625-29b4-429f-babd-3f94f96c2daa None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:54.839549 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138da625-29b4-429f-babd-3f94f96c2daa None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:54.839549 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138da625-29b4-429f-babd-3f94f96c2daa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:54.863069 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-138da625-29b4-429f-babd-3f94f96c2daa None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:09:54.864352 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-138da625-29b4-429f-babd-3f94f96c2daa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:09:54.887899 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 85/158] 10.222.0.58 () {60 vars in 1132 bytes} [Tue May 17 19:09:54 2022] POST /identity/v3/users => generated 283 bytes in 343 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 17 19:09:54.978391 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 74/159] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:09:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 17 19:09:55.021596 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360fd264-3e0d-4ee5-8435-5ed604e6d427 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:55.131011 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360fd264-3e0d-4ee5-8435-5ed604e6d427 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:55.166031 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6360c51c-521a-4ae0-ba35-e2037b0d9d35 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:55.166311 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6360c51c-521a-4ae0-ba35-e2037b0d9d35 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:55.166528 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6360c51c-521a-4ae0-ba35-e2037b0d9d35 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:55.168319 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 75/160] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:09:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 17 19:09:55.174062 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43008e93-0510-4d49-9b6c-fc2b353dd6f3 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:55.174396 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eef99f0-bcde-462d-aedf-97d7a05ddf14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:55.174514 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43008e93-0510-4d49-9b6c-fc2b353dd6f3 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:55.174665 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43008e93-0510-4d49-9b6c-fc2b353dd6f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:55.175162 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eef99f0-bcde-462d-aedf-97d7a05ddf14 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:55.175443 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eef99f0-bcde-462d-aedf-97d7a05ddf14 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:55.175752 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eef99f0-bcde-462d-aedf-97d7a05ddf14 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:55.193174 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eef99f0-bcde-462d-aedf-97d7a05ddf14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 17 19:09:55.195701 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-43008e93-0510-4d49-9b6c-fc2b353dd6f3 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:09:55.196971 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-43008e93-0510-4d49-9b6c-fc2b353dd6f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:09:55.201580 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-0eef99f0-bcde-462d-aedf-97d7a05ddf14 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 17 19:09:55.201580 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:09:55.201580 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:09:55.201580 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:09:55.201580 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:09:55.201580 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:09:55.201580 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:09:55.201580 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:09:55.201580 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:09:55.201580 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:09:55.201580 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:09:55.201580 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:09:55.201580 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 17 19:09:55.201580 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) May 17 19:09:55.201580 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 17 19:09:55.201580 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 17 19:09:55.202716 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 17 19:09:55.202716 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 17 19:09:55.202716 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 17 19:09:55.202716 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 17 19:09:55.202716 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:09:55.202716 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:09:55.202716 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:09:55.202716 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:09:55.202716 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:09:55.202716 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:09:55.202716 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:09:55.202716 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:09:55.202716 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:09:55.202716 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:09:55.202716 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:09:55.202716 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:09:55.204357 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:09:55.204357 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:09:55.204357 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:09:55.204357 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:09:55.204357 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 17 19:09:55.204357 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 17 19:09:55.204357 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 17 19:09:55.204357 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 17 19:09:55.204357 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 17 19:09:55.204357 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 17 19:09:55.204357 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 17 19:09:55.204357 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:09:55.204357 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 86/161] 10.222.0.58 () {56 vars in 1114 bytes} [Tue May 17 19:09:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:09:55.208866 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eef99f0-bcde-462d-aedf-97d7a05ddf14 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:55.301988 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 76/162] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:09:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 17 19:09:55.324372 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eef99f0-bcde-462d-aedf-97d7a05ddf14 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:55.326640 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43008e93-0510-4d49-9b6c-fc2b353dd6f3 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:55.364391 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f824296-f6d1-4f3b-b48e-90696769e0a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:55.366181 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f824296-f6d1-4f3b-b48e-90696769e0a4 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:55.366605 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f824296-f6d1-4f3b-b48e-90696769e0a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:55.367053 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f824296-f6d1-4f3b-b48e-90696769e0a4 None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:55.368026 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f824296-f6d1-4f3b-b48e-90696769e0a4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:09:55.368474 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f824296-f6d1-4f3b-b48e-90696769e0a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:55.370090 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f824296-f6d1-4f3b-b48e-90696769e0a4 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:55.380486 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 87/163] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 17 19:09:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:09:55.385996 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f824296-f6d1-4f3b-b48e-90696769e0a4 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:55.457910 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43008e93-0510-4d49-9b6c-fc2b353dd6f3 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:55.488388 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f824296-f6d1-4f3b-b48e-90696769e0a4 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:55.495567 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d91f39-50f2-4edd-966a-8dd3a268f8f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:55.496429 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d91f39-50f2-4edd-966a-8dd3a268f8f1 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:55.496750 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d91f39-50f2-4edd-966a-8dd3a268f8f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:55.497024 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d91f39-50f2-4edd-966a-8dd3a268f8f1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:55.508634 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d91f39-50f2-4edd-966a-8dd3a268f8f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 17 19:09:55.517233 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-70d91f39-50f2-4edd-966a-8dd3a268f8f1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 17 19:09:55.517233 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:09:55.517233 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:09:55.517233 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:09:55.517233 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:09:55.517233 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:09:55.517233 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:09:55.517233 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:09:55.517233 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:09:55.517233 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:09:55.517233 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:09:55.517233 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:09:55.517233 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 17 19:09:55.517233 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) May 17 19:09:55.517233 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 17 19:09:55.517233 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 17 19:09:55.518107 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 17 19:09:55.518107 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 17 19:09:55.518107 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 17 19:09:55.518107 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 17 19:09:55.518107 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:09:55.518107 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:09:55.518107 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:09:55.518107 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:09:55.518107 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:09:55.518107 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:09:55.518107 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:09:55.518107 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:09:55.518107 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:09:55.518107 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:09:55.518107 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:09:55.518107 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:09:55.518962 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:09:55.518962 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:09:55.518962 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:09:55.518962 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:09:55.518962 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 17 19:09:55.518962 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 17 19:09:55.518962 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 17 19:09:55.518962 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 17 19:09:55.518962 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 17 19:09:55.518962 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 17 19:09:55.518962 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 17 19:09:55.518962 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:09:55.518962 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 77/164] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 17 19:09:55 2022] GET /identity/v3/roles/reader => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 17 19:09:55.521903 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910b883c-302d-49d2-8ced-758a801dc009 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:55.521903 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910b883c-302d-49d2-8ced-758a801dc009 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:55.521903 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910b883c-302d-49d2-8ced-758a801dc009 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:55.522290 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 78/165] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:09:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:09:55.526106 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c320aa8-808a-41e3-9ce1-a0bc9c9ccbe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:55.526980 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c320aa8-808a-41e3-9ce1-a0bc9c9ccbe5 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:55.527260 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c320aa8-808a-41e3-9ce1-a0bc9c9ccbe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:55.527558 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c320aa8-808a-41e3-9ce1-a0bc9c9ccbe5 None admin] PATH_INFO: `/v3/users/ed3e18e472cf4c8fb0e0f8e477b961b1` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:55.528010 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910b883c-302d-49d2-8ced-758a801dc009 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:55.528783 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c320aa8-808a-41e3-9ce1-a0bc9c9ccbe5 None admin] RBAC: Authorizing `identity:get_user(user_id=ed3e18e472cf4c8fb0e0f8e477b961b1)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:55.530233 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c320aa8-808a-41e3-9ce1-a0bc9c9ccbe5 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:55.532044 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 88/166] 10.222.0.58 () {56 vars in 1177 bytes} [Tue May 17 19:09:55 2022] GET /identity/v3/users/ed3e18e472cf4c8fb0e0f8e477b961b1 => generated 275 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:09:55.533241 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d96d241-ebfb-4342-a4b5-71f6c1375dd3 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:55.533522 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d96d241-ebfb-4342-a4b5-71f6c1375dd3 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:55.533794 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d96d241-ebfb-4342-a4b5-71f6c1375dd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:55.556985 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-8d96d241-ebfb-4342-a4b5-71f6c1375dd3 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:09:55.558481 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-8d96d241-ebfb-4342-a4b5-71f6c1375dd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:09:55.644499 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 89/167] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:09:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 17 19:09:55.645534 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d96d241-ebfb-4342-a4b5-71f6c1375dd3 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:55.654218 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910b883c-302d-49d2-8ced-758a801dc009 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:55.698833 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec0454c-8941-419c-bbfb-03ee378a6bed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:55.699032 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec0454c-8941-419c-bbfb-03ee378a6bed None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:55.699277 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec0454c-8941-419c-bbfb-03ee378a6bed None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:55.699592 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec0454c-8941-419c-bbfb-03ee378a6bed None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:55.700346 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec0454c-8941-419c-bbfb-03ee378a6bed None admin] RBAC: Adding query filter params (name=reader) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:09:55.700649 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec0454c-8941-419c-bbfb-03ee378a6bed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:55.702570 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec0454c-8941-419c-bbfb-03ee378a6bed None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:55.718391 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 79/168] 10.222.0.58 () {56 vars in 1101 bytes} [Tue May 17 19:09:55 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:09:55.718391 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec0454c-8941-419c-bbfb-03ee378a6bed None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:55.766707 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d96d241-ebfb-4342-a4b5-71f6c1375dd3 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:55.801060 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d8c0c9-a31a-4e24-957a-3c83647553c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:55.802003 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d8c0c9-a31a-4e24-957a-3c83647553c4 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:55.802279 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d8c0c9-a31a-4e24-957a-3c83647553c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:55.802551 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d8c0c9-a31a-4e24-957a-3c83647553c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:55.865426 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec0454c-8941-419c-bbfb-03ee378a6bed None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:55.893982 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d8c0c9-a31a-4e24-957a-3c83647553c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:55.895437 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d8c0c9-a31a-4e24-957a-3c83647553c4 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:55.925191 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565af870-c83a-4942-9019-10040d2c328d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:55.925988 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565af870-c83a-4942-9019-10040d2c328d None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:55.926261 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565af870-c83a-4942-9019-10040d2c328d None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:55.926526 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565af870-c83a-4942-9019-10040d2c328d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:55.947368 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565af870-c83a-4942-9019-10040d2c328d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 17 19:09:55.955248 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-565af870-c83a-4942-9019-10040d2c328d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 17 19:09:55.955248 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:09:55.955248 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:09:55.955248 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:09:55.955248 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:09:55.955248 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:09:55.955248 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:09:55.955248 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:09:55.955248 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:09:55.955248 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:09:55.955248 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:09:55.955248 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:09:55.955248 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 17 19:09:55.955248 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) May 17 19:09:55.955248 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 17 19:09:55.955248 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 17 19:09:55.956148 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 17 19:09:55.956148 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 17 19:09:55.956148 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 17 19:09:55.956148 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 17 19:09:55.956148 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:09:55.956148 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:09:55.956148 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:09:55.956148 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:09:55.956148 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:09:55.956148 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:09:55.956148 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:09:55.956148 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:09:55.956148 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:09:55.956148 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:09:55.956148 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:09:55.956148 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:09:55.956921 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:09:55.956921 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:09:55.956921 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:09:55.956921 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:09:55.956921 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 17 19:09:55.956921 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 17 19:09:55.956921 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 17 19:09:55.956921 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 17 19:09:55.956921 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 17 19:09:55.956921 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 17 19:09:55.956921 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 17 19:09:55.956921 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:09:55.964838 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 80/169] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 17 19:09:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:09:55.965431 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565af870-c83a-4942-9019-10040d2c328d None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:56.019304 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 90/170] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 17 19:09:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 17 19:09:56.024240 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d8c0c9-a31a-4e24-957a-3c83647553c4 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:56.125533 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d8c0c9-a31a-4e24-957a-3c83647553c4 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:56.161929 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0cbe17-8b48-4f98-89f7-d6700feccb93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:56.161929 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0cbe17-8b48-4f98-89f7-d6700feccb93 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:56.161929 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0cbe17-8b48-4f98-89f7-d6700feccb93 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:56.161929 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0cbe17-8b48-4f98-89f7-d6700feccb93 None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:56.162318 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0cbe17-8b48-4f98-89f7-d6700feccb93 None admin] RBAC: Adding query filter params (name=member) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:09:56.162621 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0cbe17-8b48-4f98-89f7-d6700feccb93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:56.164678 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0cbe17-8b48-4f98-89f7-d6700feccb93 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:56.174189 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 91/171] 10.222.0.58 () {56 vars in 1101 bytes} [Tue May 17 19:09:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:09:56.174903 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0cbe17-8b48-4f98-89f7-d6700feccb93 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:56.183661 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565af870-c83a-4942-9019-10040d2c328d None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:56.280421 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0cbe17-8b48-4f98-89f7-d6700feccb93 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:56.335513 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adefde99-de2a-4676-ac75-beb14e839d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:56.335513 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adefde99-de2a-4676-ac75-beb14e839d7f None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:56.335513 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adefde99-de2a-4676-ac75-beb14e839d7f None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:56.335902 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adefde99-de2a-4676-ac75-beb14e839d7f None admin] PATH_INFO: `/v3/projects/d13e273646b44ecca3cd10cd867bbd04` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:56.339509 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adefde99-de2a-4676-ac75-beb14e839d7f None admin] RBAC: Authorizing `identity:get_project(project_id=d13e273646b44ecca3cd10cd867bbd04)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:56.339509 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adefde99-de2a-4676-ac75-beb14e839d7f None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:56.340512 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 92/172] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 17 19:09:56 2022] GET /identity/v3/projects/d13e273646b44ecca3cd10cd867bbd04 => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:09:56.341197 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adefde99-de2a-4676-ac75-beb14e839d7f None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:56.356827 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045467a1-e851-4d31-9f15-66415eb96145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:56.357703 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045467a1-e851-4d31-9f15-66415eb96145 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:56.357983 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045467a1-e851-4d31-9f15-66415eb96145 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:56.358257 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045467a1-e851-4d31-9f15-66415eb96145 None admin] PATH_INFO: `/v3/users/ce8e7192e8fc44da9892e1a896b57287` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:56.359555 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045467a1-e851-4d31-9f15-66415eb96145 None admin] RBAC: Authorizing `identity:get_user(user_id=ce8e7192e8fc44da9892e1a896b57287)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:56.361084 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045467a1-e851-4d31-9f15-66415eb96145 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:56.362780 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 81/173] 10.222.0.58 () {56 vars in 1177 bytes} [Tue May 17 19:09:55 2022] GET /identity/v3/users/ce8e7192e8fc44da9892e1a896b57287 => generated 289 bytes in 398 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:09:56.363575 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045467a1-e851-4d31-9f15-66415eb96145 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:56.548612 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adefde99-de2a-4676-ac75-beb14e839d7f None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:56.566705 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045467a1-e851-4d31-9f15-66415eb96145 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:56.616185 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678cc5b0-ef73-4779-956a-db43711b7e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:56.617424 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678cc5b0-ef73-4779-956a-db43711b7e08 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:56.623997 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678cc5b0-ef73-4779-956a-db43711b7e08 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:56.623997 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678cc5b0-ef73-4779-956a-db43711b7e08 None admin] PATH_INFO: `/v3/users/ed3e18e472cf4c8fb0e0f8e477b961b1` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:56.624553 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678cc5b0-ef73-4779-956a-db43711b7e08 None admin] RBAC: Authorizing `identity:get_user(user_id=ed3e18e472cf4c8fb0e0f8e477b961b1)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:56.626206 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678cc5b0-ef73-4779-956a-db43711b7e08 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:56.628107 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 93/174] 10.222.0.58 () {56 vars in 1177 bytes} [Tue May 17 19:09:56 2022] GET /identity/v3/users/ed3e18e472cf4c8fb0e0f8e477b961b1 => generated 275 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:09:56.629139 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678cc5b0-ef73-4779-956a-db43711b7e08 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:56.649868 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640c846d-af7c-411c-9c43-1973d85420e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:56.650534 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640c846d-af7c-411c-9c43-1973d85420e7 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:56.650810 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640c846d-af7c-411c-9c43-1973d85420e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:56.651078 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640c846d-af7c-411c-9c43-1973d85420e7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:56.651963 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640c846d-af7c-411c-9c43-1973d85420e7 None admin] RBAC: Adding query filter params (role.id=1055a3235b0143e58f1090c1e154e5c1, scope.project.id=d13e273646b44ecca3cd10cd867bbd04, user.id=ed3e18e472cf4c8fb0e0f8e477b961b1) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:09:56.652273 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640c846d-af7c-411c-9c43-1973d85420e7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:56.653718 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640c846d-af7c-411c-9c43-1973d85420e7 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:56.717682 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 82/175] 10.222.0.58 () {56 vars in 1374 bytes} [Tue May 17 19:09:56 2022] GET /identity/v3/role_assignments?user.id=ed3e18e472cf4c8fb0e0f8e477b961b1&scope.project.id=d13e273646b44ecca3cd10cd867bbd04&role.id=1055a3235b0143e58f1090c1e154e5c1 => generated 608 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:09:56.718854 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640c846d-af7c-411c-9c43-1973d85420e7 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:56.928474 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640c846d-af7c-411c-9c43-1973d85420e7 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:56.950353 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678cc5b0-ef73-4779-956a-db43711b7e08 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:57.011957 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c0b49c-1711-4332-be45-b80c6b490b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:57.011957 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c0b49c-1711-4332-be45-b80c6b490b27 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:57.011957 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c0b49c-1711-4332-be45-b80c6b490b27 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:57.011957 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c0b49c-1711-4332-be45-b80c6b490b27 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:57.033942 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6e76b7-4171-4b52-804f-0d34f9f07b36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:57.035171 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6e76b7-4171-4b52-804f-0d34f9f07b36 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:57.035693 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6e76b7-4171-4b52-804f-0d34f9f07b36 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:57.036162 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6e76b7-4171-4b52-804f-0d34f9f07b36 None admin] PATH_INFO: `/v3/projects/d13e273646b44ecca3cd10cd867bbd04` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:57.041813 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6e76b7-4171-4b52-804f-0d34f9f07b36 None admin] RBAC: Authorizing `identity:get_project(project_id=d13e273646b44ecca3cd10cd867bbd04)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:57.047372 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6e76b7-4171-4b52-804f-0d34f9f07b36 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:57.049454 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 94/176] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 17 19:09:56 2022] GET /identity/v3/projects/d13e273646b44ecca3cd10cd867bbd04 => generated 298 bytes in 421 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:09:57.056294 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6e76b7-4171-4b52-804f-0d34f9f07b36 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:57.130367 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c0b49c-1711-4332-be45-b80c6b490b27 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:57.133312 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c0b49c-1711-4332-be45-b80c6b490b27 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:57.252856 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 83/177] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 17 19:09:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 535 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 17 19:09:57.258841 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c0b49c-1711-4332-be45-b80c6b490b27 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:57.262475 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6e76b7-4171-4b52-804f-0d34f9f07b36 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:57.355443 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130d61ca-0191-462b-96a6-6cda9e9be0d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:57.356379 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130d61ca-0191-462b-96a6-6cda9e9be0d8 None admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:57.356656 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130d61ca-0191-462b-96a6-6cda9e9be0d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:57.356924 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130d61ca-0191-462b-96a6-6cda9e9be0d8 None admin] PATH_INFO: `/v3/projects/d13e273646b44ecca3cd10cd867bbd04/users/ce8e7192e8fc44da9892e1a896b57287/roles/8e3500b266214ae2be189641a1b4dbf1` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:57.364305 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130d61ca-0191-462b-96a6-6cda9e9be0d8 None admin] RBAC: Authorizing `identity:create_grant(project_id=d13e273646b44ecca3cd10cd867bbd04, user_id=ce8e7192e8fc44da9892e1a896b57287, role_id=8e3500b266214ae2be189641a1b4dbf1)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:57.365765 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130d61ca-0191-462b-96a6-6cda9e9be0d8 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:57.389755 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c0b49c-1711-4332-be45-b80c6b490b27 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:57.427105 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e04cbf-d97c-44c9-bdce-e439e5edb9ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:57.428012 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e04cbf-d97c-44c9-bdce-e439e5edb9ae None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:57.428289 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e04cbf-d97c-44c9-bdce-e439e5edb9ae None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:57.428564 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e04cbf-d97c-44c9-bdce-e439e5edb9ae None admin] PATH_INFO: `/v3/projects/e2de3fd8bde6430aa6a1e97114f6aa87` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:57.429835 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e04cbf-d97c-44c9-bdce-e439e5edb9ae None admin] RBAC: Authorizing `identity:get_project(project_id=e2de3fd8bde6430aa6a1e97114f6aa87)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:57.431298 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e04cbf-d97c-44c9-bdce-e439e5edb9ae None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:57.433104 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 84/178] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 17 19:09:57 2022] GET /identity/v3/projects/e2de3fd8bde6430aa6a1e97114f6aa87 => generated 312 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:09:57.438798 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e04cbf-d97c-44c9-bdce-e439e5edb9ae None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:57.455166 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 95/179] 10.222.0.58 () {58 vars in 1439 bytes} [Tue May 17 19:09:57 2022] PUT /identity/v3/projects/d13e273646b44ecca3cd10cd867bbd04/users/ce8e7192e8fc44da9892e1a896b57287/roles/8e3500b266214ae2be189641a1b4dbf1 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:09:57.557457 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e04cbf-d97c-44c9-bdce-e439e5edb9ae None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:57.605486 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3c7079-4e03-460a-b275-0d8338f9b071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:57.606932 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3c7079-4e03-460a-b275-0d8338f9b071 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:57.607450 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3c7079-4e03-460a-b275-0d8338f9b071 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:57.607962 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3c7079-4e03-460a-b275-0d8338f9b071 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:57.609003 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3c7079-4e03-460a-b275-0d8338f9b071 None admin] RBAC: Adding query filter params (role.id=114e6a16e63447d1924707ace5aace25, scope.project.id=e2de3fd8bde6430aa6a1e97114f6aa87, user.id=ed3e18e472cf4c8fb0e0f8e477b961b1) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:09:57.609509 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3c7079-4e03-460a-b275-0d8338f9b071 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:57.611169 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3c7079-4e03-460a-b275-0d8338f9b071 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:57.620733 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 85/180] 10.222.0.58 () {56 vars in 1374 bytes} [Tue May 17 19:09:57 2022] GET /identity/v3/role_assignments?user.id=ed3e18e472cf4c8fb0e0f8e477b961b1&scope.project.id=e2de3fd8bde6430aa6a1e97114f6aa87&role.id=114e6a16e63447d1924707ace5aace25 => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:09:58.031228 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea96ebf-d61c-4b73-b75f-2c5d1daf2706 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:58.031766 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea96ebf-d61c-4b73-b75f-2c5d1daf2706 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:58.031840 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea96ebf-d61c-4b73-b75f-2c5d1daf2706 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:58.033460 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 96/181] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:09:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:09:58.038408 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de46c950-ad55-4bee-b449-23d389609a0e None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:58.038727 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de46c950-ad55-4bee-b449-23d389609a0e None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:58.039010 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de46c950-ad55-4bee-b449-23d389609a0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:58.060719 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-de46c950-ad55-4bee-b449-23d389609a0e None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:09:58.061932 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-de46c950-ad55-4bee-b449-23d389609a0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:09:58.138461 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 86/182] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:09:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 17 19:09:58.161817 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea96ebf-d61c-4b73-b75f-2c5d1daf2706 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:58.217261 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bae558-be15-4c06-ba2d-bb1bd8860534 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:58.217537 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bae558-be15-4c06-ba2d-bb1bd8860534 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:58.217743 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bae558-be15-4c06-ba2d-bb1bd8860534 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:58.219445 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 87/183] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:09:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:09:58.225029 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58979bb-25c2-4d7e-9276-a7adee843914 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:58.225315 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58979bb-25c2-4d7e-9276-a7adee843914 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:58.225595 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58979bb-25c2-4d7e-9276-a7adee843914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:58.244469 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-f58979bb-25c2-4d7e-9276-a7adee843914 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:09:58.246038 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-f58979bb-25c2-4d7e-9276-a7adee843914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:09:58.284450 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea96ebf-d61c-4b73-b75f-2c5d1daf2706 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:58.318271 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50407de-03c6-4e7b-aa21-8f94fdf584f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:58.319098 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 88/184] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:09:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 17 19:09:58.320501 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b42357-0b30-4c33-b648-fc4e02ed0656 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:58.320790 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b42357-0b30-4c33-b648-fc4e02ed0656 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:58.321066 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b42357-0b30-4c33-b648-fc4e02ed0656 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:58.322461 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 89/185] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:09:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:09:58.323622 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50407de-03c6-4e7b-aa21-8f94fdf584f9 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:58.324199 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50407de-03c6-4e7b-aa21-8f94fdf584f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:58.324672 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50407de-03c6-4e7b-aa21-8f94fdf584f9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:58.327886 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9067089-3fd0-470f-a1a1-e2bfd296247c None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:58.328001 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9067089-3fd0-470f-a1a1-e2bfd296247c None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:58.328214 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9067089-3fd0-470f-a1a1-e2bfd296247c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:58.337243 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50407de-03c6-4e7b-aa21-8f94fdf584f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 17 19:09:58.345542 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-c9067089-3fd0-470f-a1a1-e2bfd296247c None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:09:58.345677 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-e50407de-03c6-4e7b-aa21-8f94fdf584f9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 17 19:09:58.345677 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:09:58.345677 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:09:58.345677 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:09:58.345677 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:09:58.345677 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:09:58.345677 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:09:58.345677 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:09:58.345677 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:09:58.345677 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:09:58.345677 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:09:58.345677 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:09:58.345677 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 17 19:09:58.345677 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) May 17 19:09:58.345677 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 17 19:09:58.345677 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 17 19:09:58.346514 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 17 19:09:58.346514 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 17 19:09:58.346514 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 17 19:09:58.346514 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 17 19:09:58.346514 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:09:58.346514 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:09:58.346514 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:09:58.346514 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:09:58.346514 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:09:58.346514 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:09:58.346514 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:09:58.346514 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:09:58.346514 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:09:58.346514 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:09:58.346514 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:09:58.346514 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:09:58.347568 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:09:58.347568 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:09:58.347568 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:09:58.347568 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:09:58.347568 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 17 19:09:58.347568 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 17 19:09:58.347568 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 17 19:09:58.347568 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 17 19:09:58.347568 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 17 19:09:58.347568 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 17 19:09:58.347568 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 17 19:09:58.347568 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:09:58.348161 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c9067089-3fd0-470f-a1a1-e2bfd296247c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:09:58.348228 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 97/186] 10.222.0.58 () {56 vars in 1096 bytes} [Tue May 17 19:09:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:09:58.348228 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50407de-03c6-4e7b-aa21-8f94fdf584f9 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:58.414517 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 90/187] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:09:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 17 19:09:58.415864 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9067089-3fd0-470f-a1a1-e2bfd296247c None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:58.450775 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50407de-03c6-4e7b-aa21-8f94fdf584f9 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:58.483759 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ce2a5d-77cd-41b2-8a96-db95fdb43461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:58.484477 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ce2a5d-77cd-41b2-8a96-db95fdb43461 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:58.484752 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ce2a5d-77cd-41b2-8a96-db95fdb43461 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:58.485016 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ce2a5d-77cd-41b2-8a96-db95fdb43461 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:58.495396 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ce2a5d-77cd-41b2-8a96-db95fdb43461 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 17 19:09:58.503820 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-14ce2a5d-77cd-41b2-8a96-db95fdb43461 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 17 19:09:58.503820 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:09:58.503820 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:09:58.503820 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:09:58.503820 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:09:58.503820 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:09:58.503820 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:09:58.503820 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:09:58.503820 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:09:58.503820 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:09:58.503820 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:09:58.503820 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:09:58.503820 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 17 19:09:58.503820 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) May 17 19:09:58.503820 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 17 19:09:58.503820 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 17 19:09:58.504724 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 17 19:09:58.504724 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 17 19:09:58.504724 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 17 19:09:58.504724 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 17 19:09:58.504724 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:09:58.504724 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:09:58.504724 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:09:58.504724 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:09:58.504724 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:09:58.504724 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:09:58.504724 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:09:58.504724 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:09:58.504724 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:09:58.504724 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:09:58.504724 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:09:58.504724 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:09:58.505504 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:09:58.505504 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:09:58.505504 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:09:58.505504 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:09:58.505504 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 17 19:09:58.505504 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 17 19:09:58.505504 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 17 19:09:58.505504 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 17 19:09:58.505504 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 17 19:09:58.505504 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 17 19:09:58.505504 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 17 19:09:58.505504 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:09:58.505504 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 98/188] 10.222.0.58 () {56 vars in 1114 bytes} [Tue May 17 19:09:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:09:58.506132 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ce2a5d-77cd-41b2-8a96-db95fdb43461 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:58.520801 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9067089-3fd0-470f-a1a1-e2bfd296247c None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:58.560306 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70a8f4e-2aae-467c-933a-75908e1476dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:58.561279 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70a8f4e-2aae-467c-933a-75908e1476dc None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:58.561557 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70a8f4e-2aae-467c-933a-75908e1476dc None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:58.562089 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70a8f4e-2aae-467c-933a-75908e1476dc None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:58.562970 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70a8f4e-2aae-467c-933a-75908e1476dc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:09:58.563284 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70a8f4e-2aae-467c-933a-75908e1476dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:58.565281 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70a8f4e-2aae-467c-933a-75908e1476dc None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:58.581005 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 91/189] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 17 19:09:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:09:58.581005 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70a8f4e-2aae-467c-933a-75908e1476dc None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:58.603627 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ce2a5d-77cd-41b2-8a96-db95fdb43461 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:58.637936 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d70d08-8f27-4109-ab47-d343f4326eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:58.638728 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d70d08-8f27-4109-ab47-d343f4326eb1 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:58.639009 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d70d08-8f27-4109-ab47-d343f4326eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:58.639281 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d70d08-8f27-4109-ab47-d343f4326eb1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:58.640472 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d70d08-8f27-4109-ab47-d343f4326eb1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:58.641984 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d70d08-8f27-4109-ab47-d343f4326eb1 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:58.647590 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 99/190] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:09:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:09:58.648987 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d70d08-8f27-4109-ab47-d343f4326eb1 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:58.681955 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70a8f4e-2aae-467c-933a-75908e1476dc None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:58.716233 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282a2e5b-02d1-429d-b146-26c9aaa36a47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:58.716892 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282a2e5b-02d1-429d-b146-26c9aaa36a47 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:58.717244 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282a2e5b-02d1-429d-b146-26c9aaa36a47 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:58.717517 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282a2e5b-02d1-429d-b146-26c9aaa36a47 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:58.718307 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282a2e5b-02d1-429d-b146-26c9aaa36a47 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:09:58.718611 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282a2e5b-02d1-429d-b146-26c9aaa36a47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:58.720001 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282a2e5b-02d1-429d-b146-26c9aaa36a47 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:58.728828 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 92/191] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 17 19:09:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:09:58.729422 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282a2e5b-02d1-429d-b146-26c9aaa36a47 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:58.749959 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d70d08-8f27-4109-ab47-d343f4326eb1 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:58.782917 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b02194-a40f-455a-8a4f-3e58b97f3457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:58.783807 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b02194-a40f-455a-8a4f-3e58b97f3457 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:58.784098 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b02194-a40f-455a-8a4f-3e58b97f3457 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:58.784357 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b02194-a40f-455a-8a4f-3e58b97f3457 None admin] PATH_INFO: `/v3/users/da56a9ca0e2e4d098b4bc6cd218c694f` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:58.797658 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b02194-a40f-455a-8a4f-3e58b97f3457 None admin] RBAC: Authorizing `identity:get_user(user_id=da56a9ca0e2e4d098b4bc6cd218c694f)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:58.799135 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b02194-a40f-455a-8a4f-3e58b97f3457 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:58.800864 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 100/192] 10.222.0.58 () {56 vars in 1177 bytes} [Tue May 17 19:09:58 2022] GET /identity/v3/users/da56a9ca0e2e4d098b4bc6cd218c694f => generated 283 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:09:58.805327 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b02194-a40f-455a-8a4f-3e58b97f3457 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:58.829830 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282a2e5b-02d1-429d-b146-26c9aaa36a47 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:58.867339 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5854f6b9-8c65-4852-85cc-c4cbb7e378b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:58.868044 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5854f6b9-8c65-4852-85cc-c4cbb7e378b4 None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:58.868325 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5854f6b9-8c65-4852-85cc-c4cbb7e378b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:58.868597 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5854f6b9-8c65-4852-85cc-c4cbb7e378b4 None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:58.869334 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5854f6b9-8c65-4852-85cc-c4cbb7e378b4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:58.870777 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5854f6b9-8c65-4852-85cc-c4cbb7e378b4 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:58.902851 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b02194-a40f-455a-8a4f-3e58b97f3457 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:58.935012 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c17326-ece3-4e2c-9c96-3284e4795fa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:58.935994 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c17326-ece3-4e2c-9c96-3284e4795fa4 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:58.936270 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c17326-ece3-4e2c-9c96-3284e4795fa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:58.936537 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c17326-ece3-4e2c-9c96-3284e4795fa4 None admin] PATH_INFO: `/v3/users/da56a9ca0e2e4d098b4bc6cd218c694f` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:58.937700 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c17326-ece3-4e2c-9c96-3284e4795fa4 None admin] RBAC: Authorizing `identity:get_user(user_id=da56a9ca0e2e4d098b4bc6cd218c694f)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:58.939131 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c17326-ece3-4e2c-9c96-3284e4795fa4 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:58.940881 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 101/193] 10.222.0.58 () {56 vars in 1177 bytes} [Tue May 17 19:09:58 2022] GET /identity/v3/users/da56a9ca0e2e4d098b4bc6cd218c694f => generated 283 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:09:58.941629 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c17326-ece3-4e2c-9c96-3284e4795fa4 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:58.979518 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 93/194] 10.222.0.58 () {60 vars in 1132 bytes} [Tue May 17 19:09:58 2022] POST /identity/v3/users => generated 297 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:09:58.979518 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5854f6b9-8c65-4852-85cc-c4cbb7e378b4 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:59.078707 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c17326-ece3-4e2c-9c96-3284e4795fa4 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:59.080982 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5854f6b9-8c65-4852-85cc-c4cbb7e378b4 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:59.117509 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893a6e8d-0e3c-4dc9-b319-9727743efdd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:59.118351 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893a6e8d-0e3c-4dc9-b319-9727743efdd9 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:59.118633 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893a6e8d-0e3c-4dc9-b319-9727743efdd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:59.118908 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893a6e8d-0e3c-4dc9-b319-9727743efdd9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:59.122084 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b6d7e3-a49b-415c-9258-5521800c4571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:59.123662 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b6d7e3-a49b-415c-9258-5521800c4571 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:59.123938 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b6d7e3-a49b-415c-9258-5521800c4571 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:59.124201 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b6d7e3-a49b-415c-9258-5521800c4571 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:59.197044 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893a6e8d-0e3c-4dc9-b319-9727743efdd9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:59.198592 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893a6e8d-0e3c-4dc9-b319-9727743efdd9 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:59.210029 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b6d7e3-a49b-415c-9258-5521800c4571 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:59.212400 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b6d7e3-a49b-415c-9258-5521800c4571 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:59.331253 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 102/195] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 17 19:09:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 17 19:09:59.338497 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b6d7e3-a49b-415c-9258-5521800c4571 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:59.339534 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 94/196] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 17 19:09:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 17 19:09:59.345777 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893a6e8d-0e3c-4dc9-b319-9727743efdd9 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:59.443291 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b6d7e3-a49b-415c-9258-5521800c4571 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:59.463737 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893a6e8d-0e3c-4dc9-b319-9727743efdd9 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:59.510333 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6da460-5370-476a-90dc-eaecde25bce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:59.511205 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6da460-5370-476a-90dc-eaecde25bce3 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:59.511509 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6da460-5370-476a-90dc-eaecde25bce3 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:59.511785 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6da460-5370-476a-90dc-eaecde25bce3 None admin] PATH_INFO: `/v3/projects/625d29bdd4ad47ee92370dc3404fe414` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:59.513611 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6da460-5370-476a-90dc-eaecde25bce3 None admin] RBAC: Authorizing `identity:get_project(project_id=625d29bdd4ad47ee92370dc3404fe414)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:59.515076 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6da460-5370-476a-90dc-eaecde25bce3 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:59.517001 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 103/197] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 17 19:09:59 2022] GET /identity/v3/projects/625d29bdd4ad47ee92370dc3404fe414 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:09:59.524552 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6da460-5370-476a-90dc-eaecde25bce3 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:59.530653 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94291122-50cc-485f-857c-2d1798a09e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:59.531842 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94291122-50cc-485f-857c-2d1798a09e90 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:59.536583 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94291122-50cc-485f-857c-2d1798a09e90 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:59.537096 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94291122-50cc-485f-857c-2d1798a09e90 None admin] PATH_INFO: `/v3/projects/625d29bdd4ad47ee92370dc3404fe414` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:59.540321 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94291122-50cc-485f-857c-2d1798a09e90 None admin] RBAC: Authorizing `identity:get_project(project_id=625d29bdd4ad47ee92370dc3404fe414)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:59.542017 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94291122-50cc-485f-857c-2d1798a09e90 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:59.544314 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 95/198] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 17 19:09:59 2022] GET /identity/v3/projects/625d29bdd4ad47ee92370dc3404fe414 => generated 302 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:09:59.552409 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94291122-50cc-485f-857c-2d1798a09e90 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:09:59.657679 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6da460-5370-476a-90dc-eaecde25bce3 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:59.720211 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da75f3f4-3a29-41c9-9813-2ec2a09e010d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:59.721190 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da75f3f4-3a29-41c9-9813-2ec2a09e010d None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:59.721636 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da75f3f4-3a29-41c9-9813-2ec2a09e010d None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:59.722322 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da75f3f4-3a29-41c9-9813-2ec2a09e010d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:59.723320 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da75f3f4-3a29-41c9-9813-2ec2a09e010d None admin] RBAC: Adding query filter params (role.id=60af5088793e4379b88821a86552a344, scope.project.id=625d29bdd4ad47ee92370dc3404fe414, user.id=da56a9ca0e2e4d098b4bc6cd218c694f) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:09:59.724782 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da75f3f4-3a29-41c9-9813-2ec2a09e010d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:59.727692 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da75f3f4-3a29-41c9-9813-2ec2a09e010d None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:59.737373 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 104/199] 10.222.0.58 () {56 vars in 1374 bytes} [Tue May 17 19:09:59 2022] GET /identity/v3/role_assignments?user.id=da56a9ca0e2e4d098b4bc6cd218c694f&scope.project.id=625d29bdd4ad47ee92370dc3404fe414&role.id=60af5088793e4379b88821a86552a344 => generated 259 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:09:59.752421 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94291122-50cc-485f-857c-2d1798a09e90 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:09:59.793726 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2792db3d-4efe-4e50-b960-4a5c7b26916d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:09:59.794405 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2792db3d-4efe-4e50-b960-4a5c7b26916d None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:09:59.794683 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2792db3d-4efe-4e50-b960-4a5c7b26916d None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:09:59.794952 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2792db3d-4efe-4e50-b960-4a5c7b26916d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:09:59.795836 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2792db3d-4efe-4e50-b960-4a5c7b26916d None admin] RBAC: Adding query filter params (role.id=1055a3235b0143e58f1090c1e154e5c1, scope.project.id=625d29bdd4ad47ee92370dc3404fe414, user.id=da56a9ca0e2e4d098b4bc6cd218c694f) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:09:59.796144 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2792db3d-4efe-4e50-b960-4a5c7b26916d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:09:59.797651 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2792db3d-4efe-4e50-b960-4a5c7b26916d None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:09:59.809762 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 96/200] 10.222.0.58 () {56 vars in 1374 bytes} [Tue May 17 19:09:59 2022] GET /identity/v3/role_assignments?user.id=da56a9ca0e2e4d098b4bc6cd218c694f&scope.project.id=625d29bdd4ad47ee92370dc3404fe414&role.id=1055a3235b0143e58f1090c1e154e5c1 => generated 259 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:00.039928 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c477ba-1abc-4565-a571-55e7863ca156 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:00.040269 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c477ba-1abc-4565-a571-55e7863ca156 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:00.040479 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c477ba-1abc-4565-a571-55e7863ca156 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:00.041866 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 105/201] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:10:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:10:00.047356 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3416a1-eefe-42b4-9965-3f16e4976be0 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:00.047735 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3416a1-eefe-42b4-9965-3f16e4976be0 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:00.048014 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3416a1-eefe-42b4-9965-3f16e4976be0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:00.069320 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-ba3416a1-eefe-42b4-9965-3f16e4976be0 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:10:00.071111 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-ba3416a1-eefe-42b4-9965-3f16e4976be0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:10:00.145267 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 97/202] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:10:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 17 19:10:00.183771 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c477ba-1abc-4565-a571-55e7863ca156 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:00.303108 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c477ba-1abc-4565-a571-55e7863ca156 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:00.337418 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d5b5c9-233a-442e-b6cc-9c820e3710ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:00.338214 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d5b5c9-233a-442e-b6cc-9c820e3710ab None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:00.338499 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d5b5c9-233a-442e-b6cc-9c820e3710ab None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:00.338771 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d5b5c9-233a-442e-b6cc-9c820e3710ab None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:00.349325 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d5b5c9-233a-442e-b6cc-9c820e3710ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 17 19:10:00.357291 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-50d5b5c9-233a-442e-b6cc-9c820e3710ab None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 17 19:10:00.357291 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:00.357291 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:00.357291 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:00.357291 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:00.357291 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:00.357291 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:00.357291 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:00.357291 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:00.357291 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:00.357291 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:00.357291 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:00.357291 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 17 19:10:00.357291 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) May 17 19:10:00.357291 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 17 19:10:00.357291 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 17 19:10:00.358197 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 17 19:10:00.358197 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 17 19:10:00.358197 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 17 19:10:00.358197 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 17 19:10:00.358197 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:00.358197 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:00.358197 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:00.358197 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:00.358197 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:00.358197 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:00.358197 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:00.358197 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:10:00.358197 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:00.358197 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:00.358197 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:00.358197 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:00.358980 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:00.358980 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:00.358980 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:00.358980 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:00.358980 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 17 19:10:00.358980 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 17 19:10:00.358980 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 17 19:10:00.358980 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 17 19:10:00.358980 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 17 19:10:00.358980 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 17 19:10:00.358980 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 17 19:10:00.358980 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:10:00.358980 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 106/203] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 17 19:10:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:10:00.366008 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3416a1-eefe-42b4-9965-3f16e4976be0 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:00.470862 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3416a1-eefe-42b4-9965-3f16e4976be0 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:00.505602 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fcb89d-ad15-4e4f-bda8-069451309531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:00.506299 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fcb89d-ad15-4e4f-bda8-069451309531 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:00.506579 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fcb89d-ad15-4e4f-bda8-069451309531 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:00.506857 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fcb89d-ad15-4e4f-bda8-069451309531 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:00.508239 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fcb89d-ad15-4e4f-bda8-069451309531 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:00.508546 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fcb89d-ad15-4e4f-bda8-069451309531 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:00.509947 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fcb89d-ad15-4e4f-bda8-069451309531 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:00.519422 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 98/204] 10.222.0.58 () {56 vars in 1101 bytes} [Tue May 17 19:10:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:00.525025 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d5b5c9-233a-442e-b6cc-9c820e3710ab None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:00.639664 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d5b5c9-233a-442e-b6cc-9c820e3710ab None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:00.673714 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195299a4-4e74-4aee-a7fb-284964a70382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:00.674765 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195299a4-4e74-4aee-a7fb-284964a70382 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:00.674930 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195299a4-4e74-4aee-a7fb-284964a70382 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:00.675203 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195299a4-4e74-4aee-a7fb-284964a70382 None admin] PATH_INFO: `/v3/users/ce8e7192e8fc44da9892e1a896b57287` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:00.676460 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195299a4-4e74-4aee-a7fb-284964a70382 None admin] RBAC: Authorizing `identity:get_user(user_id=ce8e7192e8fc44da9892e1a896b57287)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:00.677931 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195299a4-4e74-4aee-a7fb-284964a70382 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:00.680177 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 107/205] 10.222.0.58 () {56 vars in 1177 bytes} [Tue May 17 19:10:00 2022] GET /identity/v3/users/ce8e7192e8fc44da9892e1a896b57287 => generated 289 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:00.687717 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fcb89d-ad15-4e4f-bda8-069451309531 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:00.793072 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fcb89d-ad15-4e4f-bda8-069451309531 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:00.827817 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf429852-aa99-4f5e-877b-0384a6f7aa98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:00.828528 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf429852-aa99-4f5e-877b-0384a6f7aa98 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:00.828808 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf429852-aa99-4f5e-877b-0384a6f7aa98 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:00.829131 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf429852-aa99-4f5e-877b-0384a6f7aa98 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:00.898926 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf429852-aa99-4f5e-877b-0384a6f7aa98 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:00.900387 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf429852-aa99-4f5e-877b-0384a6f7aa98 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:00.999536 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 99/206] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 17 19:10:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 17 19:10:01.005086 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195299a4-4e74-4aee-a7fb-284964a70382 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:01.112840 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195299a4-4e74-4aee-a7fb-284964a70382 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:01.148340 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6a8a9c-5e60-40c0-9ca5-b7cde3e4e73e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:01.149267 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6a8a9c-5e60-40c0-9ca5-b7cde3e4e73e None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:01.149490 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6a8a9c-5e60-40c0-9ca5-b7cde3e4e73e None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:01.149765 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6a8a9c-5e60-40c0-9ca5-b7cde3e4e73e None admin] PATH_INFO: `/v3/projects/d13e273646b44ecca3cd10cd867bbd04` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:01.151032 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6a8a9c-5e60-40c0-9ca5-b7cde3e4e73e None admin] RBAC: Authorizing `identity:get_project(project_id=d13e273646b44ecca3cd10cd867bbd04)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:01.152584 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6a8a9c-5e60-40c0-9ca5-b7cde3e4e73e None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:01.154495 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 108/207] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 17 19:10:01 2022] GET /identity/v3/projects/d13e273646b44ecca3cd10cd867bbd04 => generated 298 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:01.160265 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf429852-aa99-4f5e-877b-0384a6f7aa98 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:01.262333 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf429852-aa99-4f5e-877b-0384a6f7aa98 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:01.298167 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5368949a-f395-440d-8821-c6e3b2adcb76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:01.298827 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5368949a-f395-440d-8821-c6e3b2adcb76 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:01.299106 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5368949a-f395-440d-8821-c6e3b2adcb76 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:01.299380 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5368949a-f395-440d-8821-c6e3b2adcb76 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:01.300243 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5368949a-f395-440d-8821-c6e3b2adcb76 None admin] RBAC: Adding query filter params (role.id=8e3500b266214ae2be189641a1b4dbf1, scope.project.id=d13e273646b44ecca3cd10cd867bbd04, user.id=ce8e7192e8fc44da9892e1a896b57287) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:01.300549 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5368949a-f395-440d-8821-c6e3b2adcb76 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:01.302013 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5368949a-f395-440d-8821-c6e3b2adcb76 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:01.308451 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 100/208] 10.222.0.58 () {56 vars in 1374 bytes} [Tue May 17 19:10:01 2022] GET /identity/v3/role_assignments?user.id=ce8e7192e8fc44da9892e1a896b57287&scope.project.id=d13e273646b44ecca3cd10cd867bbd04&role.id=8e3500b266214ae2be189641a1b4dbf1 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:02.078681 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa34811a-2a44-4040-9178-685da8b49d32 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:02.079188 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa34811a-2a44-4040-9178-685da8b49d32 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:02.079264 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa34811a-2a44-4040-9178-685da8b49d32 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:02.080860 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 109/209] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:10:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:10:02.086276 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cf683c-fd7f-4d98-b5d2-8879640d358c None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:02.086659 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cf683c-fd7f-4d98-b5d2-8879640d358c None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:02.086820 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cf683c-fd7f-4d98-b5d2-8879640d358c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:02.104362 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-b9cf683c-fd7f-4d98-b5d2-8879640d358c None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:10:02.105558 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b9cf683c-fd7f-4d98-b5d2-8879640d358c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:10:02.109330 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0324fc-0394-4668-8f0f-c756493a546b None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:02.109614 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0324fc-0394-4668-8f0f-c756493a546b None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:02.109887 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0324fc-0394-4668-8f0f-c756493a546b None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:02.111365 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 110/210] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:10:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 17 19:10:02.116463 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189f4a6f-03e9-46f1-a516-7f2b78011d5a None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:02.117238 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189f4a6f-03e9-46f1-a516-7f2b78011d5a None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:02.117513 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189f4a6f-03e9-46f1-a516-7f2b78011d5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:02.135239 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-189f4a6f-03e9-46f1-a516-7f2b78011d5a None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:10:02.136750 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-189f4a6f-03e9-46f1-a516-7f2b78011d5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:10:02.187300 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 101/211] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:10:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 17 19:10:02.211476 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 111/212] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:10:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 17 19:10:02.219725 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cf683c-fd7f-4d98-b5d2-8879640d358c None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:02.235651 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189f4a6f-03e9-46f1-a516-7f2b78011d5a None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:02.314624 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cf683c-fd7f-4d98-b5d2-8879640d358c None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:02.337122 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189f4a6f-03e9-46f1-a516-7f2b78011d5a None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:02.349680 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13fef6f-2c10-4338-8b0c-99adff9a55cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:02.350467 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13fef6f-2c10-4338-8b0c-99adff9a55cd None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:02.350776 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13fef6f-2c10-4338-8b0c-99adff9a55cd None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:02.351051 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13fef6f-2c10-4338-8b0c-99adff9a55cd None admin] PATH_INFO: `/v3/domains/default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:02.352234 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13fef6f-2c10-4338-8b0c-99adff9a55cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:02.353818 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13fef6f-2c10-4338-8b0c-99adff9a55cd None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:02.355302 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 102/213] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:10:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:10:02.360965 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13fef6f-2c10-4338-8b0c-99adff9a55cd None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:02.376299 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bf5bca-0c30-4ed8-bd6b-44601e5e6263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:02.377099 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bf5bca-0c30-4ed8-bd6b-44601e5e6263 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:02.377376 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bf5bca-0c30-4ed8-bd6b-44601e5e6263 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:02.377703 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bf5bca-0c30-4ed8-bd6b-44601e5e6263 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:02.388172 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bf5bca-0c30-4ed8-bd6b-44601e5e6263 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 17 19:10:02.395758 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-23bf5bca-0c30-4ed8-bd6b-44601e5e6263 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 17 19:10:02.395758 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:02.395758 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:02.395758 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:02.395758 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:02.395758 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:02.395758 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:02.395758 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:02.395758 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:02.395758 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:02.395758 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:02.395758 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:02.395758 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 17 19:10:02.395758 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) May 17 19:10:02.395758 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 17 19:10:02.395758 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 17 19:10:02.396657 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 17 19:10:02.396657 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 17 19:10:02.396657 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 17 19:10:02.396657 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 17 19:10:02.396657 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:02.396657 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:02.396657 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:02.396657 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:02.396657 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:02.396657 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:02.396657 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:02.396657 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:10:02.396657 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:02.396657 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:02.396657 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:02.396657 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:02.397425 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:02.397425 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:02.397425 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:02.397425 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:02.397425 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 17 19:10:02.397425 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 17 19:10:02.397425 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 17 19:10:02.397425 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 17 19:10:02.397425 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 17 19:10:02.397425 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 17 19:10:02.397425 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 17 19:10:02.397425 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:10:02.397425 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 112/214] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 17 19:10:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 17 19:10:02.399158 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64298407-a9b8-4a5d-b213-e376c271d485 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:02.399440 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64298407-a9b8-4a5d-b213-e376c271d485 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:02.399735 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64298407-a9b8-4a5d-b213-e376c271d485 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:02.401088 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 113/215] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:10:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:10:02.402276 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d60f070-fd3b-476e-85f4-de87b0280396 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:02.402562 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d60f070-fd3b-476e-85f4-de87b0280396 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:02.402835 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d60f070-fd3b-476e-85f4-de87b0280396 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:02.404226 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 114/216] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:10:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:10:02.404954 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d60f070-fd3b-476e-85f4-de87b0280396 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:02.462084 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13fef6f-2c10-4338-8b0c-99adff9a55cd None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:02.494139 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdbe4c2-8a80-4db5-93ca-01e5fb672fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:02.494803 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdbe4c2-8a80-4db5-93ca-01e5fb672fe5 None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:02.495079 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdbe4c2-8a80-4db5-93ca-01e5fb672fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:02.495349 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdbe4c2-8a80-4db5-93ca-01e5fb672fe5 None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:02.496085 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdbe4c2-8a80-4db5-93ca-01e5fb672fe5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:02.497519 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdbe4c2-8a80-4db5-93ca-01e5fb672fe5 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:02.499736 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d60f070-fd3b-476e-85f4-de87b0280396 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:02.531078 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be248509-e78c-40b4-8561-a40c25861a23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:02.531757 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be248509-e78c-40b4-8561-a40c25861a23 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:02.532035 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be248509-e78c-40b4-8561-a40c25861a23 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:02.532308 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be248509-e78c-40b4-8561-a40c25861a23 None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:02.533040 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be248509-e78c-40b4-8561-a40c25861a23 None admin] RBAC: Adding query filter params (name=member) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:02.533343 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be248509-e78c-40b4-8561-a40c25861a23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:02.534713 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be248509-e78c-40b4-8561-a40c25861a23 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:02.543295 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 115/217] 10.222.0.58 () {56 vars in 1101 bytes} [Tue May 17 19:10:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:02.544591 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafcacb9-a2c1-4b7c-8623-a6c37e395939 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:02.544877 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafcacb9-a2c1-4b7c-8623-a6c37e395939 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:02.545148 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafcacb9-a2c1-4b7c-8623-a6c37e395939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:02.560011 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-bafcacb9-a2c1-4b7c-8623-a6c37e395939 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:10:02.561172 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-bafcacb9-a2c1-4b7c-8623-a6c37e395939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:10:02.622255 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 116/218] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:10:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 17 19:10:02.623594 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf3b18c-1016-45e5-acd8-d77c1942a72a None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:02.623897 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf3b18c-1016-45e5-acd8-d77c1942a72a None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:02.624157 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf3b18c-1016-45e5-acd8-d77c1942a72a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:02.639401 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-cbf3b18c-1016-45e5-acd8-d77c1942a72a None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:10:02.640643 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-cbf3b18c-1016-45e5-acd8-d77c1942a72a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:10:02.647701 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 103/219] 10.222.0.58 () {60 vars in 1132 bytes} [Tue May 17 19:10:02 2022] POST /identity/v3/users => generated 297 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 17 19:10:02.648607 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdbe4c2-8a80-4db5-93ca-01e5fb672fe5 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:02.725017 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 117/220] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:10:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 17 19:10:02.725017 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf3b18c-1016-45e5-acd8-d77c1942a72a None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:02.764712 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdbe4c2-8a80-4db5-93ca-01e5fb672fe5 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:02.798686 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecbdd51-81fc-40ca-a2fa-a993df8a94ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:02.799606 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecbdd51-81fc-40ca-a2fa-a993df8a94ad None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:02.799884 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecbdd51-81fc-40ca-a2fa-a993df8a94ad None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:02.800152 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecbdd51-81fc-40ca-a2fa-a993df8a94ad None admin] PATH_INFO: `/v3/users/614688dc21d94bc7a5f85a06df453136` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:02.815337 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecbdd51-81fc-40ca-a2fa-a993df8a94ad None admin] RBAC: Authorizing `identity:get_user(user_id=614688dc21d94bc7a5f85a06df453136)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:02.816896 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecbdd51-81fc-40ca-a2fa-a993df8a94ad None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:02.818826 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 104/221] 10.222.0.58 () {56 vars in 1177 bytes} [Tue May 17 19:10:02 2022] GET /identity/v3/users/614688dc21d94bc7a5f85a06df453136 => generated 297 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:02.819998 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecbdd51-81fc-40ca-a2fa-a993df8a94ad None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:02.832728 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf3b18c-1016-45e5-acd8-d77c1942a72a None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:02.870175 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2ae5da-e850-4fe6-b5e5-4d13905266e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:02.870926 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2ae5da-e850-4fe6-b5e5-4d13905266e2 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:02.871204 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2ae5da-e850-4fe6-b5e5-4d13905266e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:02.871472 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2ae5da-e850-4fe6-b5e5-4d13905266e2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:02.886583 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2ae5da-e850-4fe6-b5e5-4d13905266e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 17 19:10:02.894648 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-3a2ae5da-e850-4fe6-b5e5-4d13905266e2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 17 19:10:02.894648 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:02.894648 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:02.894648 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:02.894648 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:02.894648 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:02.894648 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:02.894648 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:02.894648 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:02.894648 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:02.894648 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:02.894648 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:02.894648 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 17 19:10:02.894648 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) May 17 19:10:02.894648 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 17 19:10:02.894648 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 17 19:10:02.895766 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 17 19:10:02.895766 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 17 19:10:02.895766 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 17 19:10:02.895766 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 17 19:10:02.895766 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:02.895766 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:02.895766 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:02.895766 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:02.895766 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:02.895766 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:02.895766 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:02.895766 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:10:02.895766 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:02.895766 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:02.895766 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:02.895766 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:02.896672 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:02.896672 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:02.896672 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:02.896672 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:02.896672 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 17 19:10:02.896672 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 17 19:10:02.896672 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 17 19:10:02.896672 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 17 19:10:02.896672 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 17 19:10:02.896672 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 17 19:10:02.896672 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 17 19:10:02.896672 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:10:02.896672 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 118/222] 10.222.0.58 () {56 vars in 1114 bytes} [Tue May 17 19:10:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:10:02.897449 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2ae5da-e850-4fe6-b5e5-4d13905266e2 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:02.935958 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecbdd51-81fc-40ca-a2fa-a993df8a94ad None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:02.979873 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7109af08-4af2-44d5-b988-e2ce15a946b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:02.980779 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7109af08-4af2-44d5-b988-e2ce15a946b4 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:02.981063 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7109af08-4af2-44d5-b988-e2ce15a946b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:02.981335 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7109af08-4af2-44d5-b988-e2ce15a946b4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:02.995967 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7109af08-4af2-44d5-b988-e2ce15a946b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 17 19:10:03.005631 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-7109af08-4af2-44d5-b988-e2ce15a946b4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 17 19:10:03.005631 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:03.005631 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:03.005631 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:03.005631 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:03.005631 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:03.005631 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:03.005631 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:03.005631 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:03.005631 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:03.005631 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:03.005631 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:03.005631 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 17 19:10:03.005631 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) May 17 19:10:03.005631 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 17 19:10:03.005631 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 17 19:10:03.006578 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 17 19:10:03.006578 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 17 19:10:03.006578 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 17 19:10:03.006578 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 17 19:10:03.006578 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:03.006578 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:03.006578 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:03.006578 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:03.006578 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:03.006578 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:03.006578 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:03.006578 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:10:03.006578 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:03.006578 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:03.006578 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:03.006578 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:03.007427 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:03.007427 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:03.007427 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:03.007427 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:03.007427 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 17 19:10:03.007427 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 17 19:10:03.007427 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 17 19:10:03.007427 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 17 19:10:03.007427 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 17 19:10:03.007427 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 17 19:10:03.007427 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 17 19:10:03.007427 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:10:03.007427 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 105/223] 10.222.0.58 () {56 vars in 1096 bytes} [Tue May 17 19:10:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:10:03.008107 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7109af08-4af2-44d5-b988-e2ce15a946b4 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:03.026249 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2ae5da-e850-4fe6-b5e5-4d13905266e2 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:03.069886 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a91795-6ba7-4eb3-8b6f-52bb8012f7ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:03.070529 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a91795-6ba7-4eb3-8b6f-52bb8012f7ac None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:03.070803 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a91795-6ba7-4eb3-8b6f-52bb8012f7ac None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:03.071072 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a91795-6ba7-4eb3-8b6f-52bb8012f7ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:03.120860 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7109af08-4af2-44d5-b988-e2ce15a946b4 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:03.140386 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a91795-6ba7-4eb3-8b6f-52bb8012f7ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:03.141870 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a91795-6ba7-4eb3-8b6f-52bb8012f7ac None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:03.156976 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d52c40-dd3d-4374-ac57-82cadebbf674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:03.157977 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d52c40-dd3d-4374-ac57-82cadebbf674 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:03.158457 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d52c40-dd3d-4374-ac57-82cadebbf674 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:03.158929 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d52c40-dd3d-4374-ac57-82cadebbf674 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:03.159919 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d52c40-dd3d-4374-ac57-82cadebbf674 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:03.160422 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d52c40-dd3d-4374-ac57-82cadebbf674 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:03.162021 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d52c40-dd3d-4374-ac57-82cadebbf674 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:03.171556 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 106/224] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 17 19:10:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:03.172707 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d52c40-dd3d-4374-ac57-82cadebbf674 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:03.248064 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 119/225] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 17 19:10:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 17 19:10:03.248688 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a91795-6ba7-4eb3-8b6f-52bb8012f7ac None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:03.275088 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d52c40-dd3d-4374-ac57-82cadebbf674 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:03.311079 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a958577-50a0-499d-99d5-f2865a613732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:03.312163 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a958577-50a0-499d-99d5-f2865a613732 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:03.312651 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a958577-50a0-499d-99d5-f2865a613732 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:03.313117 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a958577-50a0-499d-99d5-f2865a613732 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:03.314062 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a958577-50a0-499d-99d5-f2865a613732 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:03.314587 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a958577-50a0-499d-99d5-f2865a613732 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:03.316203 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a958577-50a0-499d-99d5-f2865a613732 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:03.332403 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 107/226] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 17 19:10:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:03.333527 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a958577-50a0-499d-99d5-f2865a613732 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:03.352308 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a91795-6ba7-4eb3-8b6f-52bb8012f7ac None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:03.387274 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393bf96b-8399-4554-8c23-08fd5f0b5f6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:03.388301 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393bf96b-8399-4554-8c23-08fd5f0b5f6b None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:03.388582 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393bf96b-8399-4554-8c23-08fd5f0b5f6b None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:03.388857 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393bf96b-8399-4554-8c23-08fd5f0b5f6b None admin] PATH_INFO: `/v3/users/da56a9ca0e2e4d098b4bc6cd218c694f` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:03.390054 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393bf96b-8399-4554-8c23-08fd5f0b5f6b None admin] RBAC: Authorizing `identity:get_user(user_id=da56a9ca0e2e4d098b4bc6cd218c694f)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:03.391512 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393bf96b-8399-4554-8c23-08fd5f0b5f6b None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:03.397696 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 120/227] 10.222.0.58 () {56 vars in 1177 bytes} [Tue May 17 19:10:03 2022] GET /identity/v3/users/da56a9ca0e2e4d098b4bc6cd218c694f => generated 283 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:03.398704 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393bf96b-8399-4554-8c23-08fd5f0b5f6b None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:03.436478 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a958577-50a0-499d-99d5-f2865a613732 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:03.470440 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd402b9-4d98-45cd-8779-2ac8545cdfdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:03.471698 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd402b9-4d98-45cd-8779-2ac8545cdfdc None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:03.472430 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd402b9-4d98-45cd-8779-2ac8545cdfdc None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:03.472896 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd402b9-4d98-45cd-8779-2ac8545cdfdc None admin] PATH_INFO: `/v3/projects/625d29bdd4ad47ee92370dc3404fe414` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:03.474364 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd402b9-4d98-45cd-8779-2ac8545cdfdc None admin] RBAC: Authorizing `identity:get_project(project_id=625d29bdd4ad47ee92370dc3404fe414)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:03.476100 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd402b9-4d98-45cd-8779-2ac8545cdfdc None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:03.478327 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 108/228] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 17 19:10:03 2022] GET /identity/v3/projects/625d29bdd4ad47ee92370dc3404fe414 => generated 302 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:03.479383 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd402b9-4d98-45cd-8779-2ac8545cdfdc None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:03.498819 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393bf96b-8399-4554-8c23-08fd5f0b5f6b None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:03.533233 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffe142a-13d5-43a2-84ac-9787a138f226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:03.533964 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffe142a-13d5-43a2-84ac-9787a138f226 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:03.534246 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffe142a-13d5-43a2-84ac-9787a138f226 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:03.534517 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffe142a-13d5-43a2-84ac-9787a138f226 None admin] PATH_INFO: `/v3/users/da56a9ca0e2e4d098b4bc6cd218c694f` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:03.536084 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffe142a-13d5-43a2-84ac-9787a138f226 None admin] RBAC: Authorizing `identity:get_user(user_id=da56a9ca0e2e4d098b4bc6cd218c694f)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:03.538027 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffe142a-13d5-43a2-84ac-9787a138f226 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:03.540695 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 121/229] 10.222.0.58 () {56 vars in 1177 bytes} [Tue May 17 19:10:03 2022] GET /identity/v3/users/da56a9ca0e2e4d098b4bc6cd218c694f => generated 283 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:03.543930 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffe142a-13d5-43a2-84ac-9787a138f226 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:03.589443 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd402b9-4d98-45cd-8779-2ac8545cdfdc None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:03.632876 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabb78d4-eb84-42a5-9b12-86ad54469da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:03.633804 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabb78d4-eb84-42a5-9b12-86ad54469da1 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:03.634088 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabb78d4-eb84-42a5-9b12-86ad54469da1 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:03.634366 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabb78d4-eb84-42a5-9b12-86ad54469da1 None admin] PATH_INFO: `/v3/projects/625d29bdd4ad47ee92370dc3404fe414` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:03.635688 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabb78d4-eb84-42a5-9b12-86ad54469da1 None admin] RBAC: Authorizing `identity:get_project(project_id=625d29bdd4ad47ee92370dc3404fe414)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:03.637146 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabb78d4-eb84-42a5-9b12-86ad54469da1 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:03.639522 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 109/230] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 17 19:10:03 2022] GET /identity/v3/projects/625d29bdd4ad47ee92370dc3404fe414 => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:03.640207 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabb78d4-eb84-42a5-9b12-86ad54469da1 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:03.677054 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffe142a-13d5-43a2-84ac-9787a138f226 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:03.717696 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929a4bf8-a9df-4afd-a526-e68bd10112d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:03.719052 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929a4bf8-a9df-4afd-a526-e68bd10112d2 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:03.719366 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929a4bf8-a9df-4afd-a526-e68bd10112d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:03.719693 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929a4bf8-a9df-4afd-a526-e68bd10112d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:03.720531 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929a4bf8-a9df-4afd-a526-e68bd10112d2 None admin] RBAC: Adding query filter params (role.id=114e6a16e63447d1924707ace5aace25, scope.project.id=625d29bdd4ad47ee92370dc3404fe414, user.id=614688dc21d94bc7a5f85a06df453136) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:03.720842 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929a4bf8-a9df-4afd-a526-e68bd10112d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:03.722547 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929a4bf8-a9df-4afd-a526-e68bd10112d2 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:03.729506 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 122/231] 10.222.0.58 () {56 vars in 1374 bytes} [Tue May 17 19:10:03 2022] GET /identity/v3/role_assignments?user.id=614688dc21d94bc7a5f85a06df453136&scope.project.id=625d29bdd4ad47ee92370dc3404fe414&role.id=114e6a16e63447d1924707ace5aace25 => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:03.730099 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929a4bf8-a9df-4afd-a526-e68bd10112d2 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:03.763942 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabb78d4-eb84-42a5-9b12-86ad54469da1 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:03.800420 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bada2f-411e-4006-9e7f-2df76a21a931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:03.801322 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bada2f-411e-4006-9e7f-2df76a21a931 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:03.801647 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bada2f-411e-4006-9e7f-2df76a21a931 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:03.801982 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bada2f-411e-4006-9e7f-2df76a21a931 None admin] PATH_INFO: `/v3/projects/625d29bdd4ad47ee92370dc3404fe414` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:03.803168 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bada2f-411e-4006-9e7f-2df76a21a931 None admin] RBAC: Authorizing `identity:get_project(project_id=625d29bdd4ad47ee92370dc3404fe414)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:03.805296 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bada2f-411e-4006-9e7f-2df76a21a931 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:03.807098 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 110/232] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 17 19:10:03 2022] GET /identity/v3/projects/625d29bdd4ad47ee92370dc3404fe414 => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:03.819524 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bada2f-411e-4006-9e7f-2df76a21a931 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:03.882459 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929a4bf8-a9df-4afd-a526-e68bd10112d2 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:03.936535 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33149f11-d7ad-4d78-91a2-5a81b83cd7b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:03.937677 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33149f11-d7ad-4d78-91a2-5a81b83cd7b9 None admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:03.937966 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33149f11-d7ad-4d78-91a2-5a81b83cd7b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:03.938241 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33149f11-d7ad-4d78-91a2-5a81b83cd7b9 None admin] PATH_INFO: `/v3/projects/625d29bdd4ad47ee92370dc3404fe414/users/da56a9ca0e2e4d098b4bc6cd218c694f/roles/1055a3235b0143e58f1090c1e154e5c1` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:03.941176 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33149f11-d7ad-4d78-91a2-5a81b83cd7b9 None admin] RBAC: Authorizing `identity:create_grant(project_id=625d29bdd4ad47ee92370dc3404fe414, user_id=da56a9ca0e2e4d098b4bc6cd218c694f, role_id=1055a3235b0143e58f1090c1e154e5c1)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:03.943321 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33149f11-d7ad-4d78-91a2-5a81b83cd7b9 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:03.998555 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bada2f-411e-4006-9e7f-2df76a21a931 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:04.019514 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 123/233] 10.222.0.58 () {58 vars in 1439 bytes} [Tue May 17 19:10:03 2022] PUT /identity/v3/projects/625d29bdd4ad47ee92370dc3404fe414/users/da56a9ca0e2e4d098b4bc6cd218c694f/roles/1055a3235b0143e58f1090c1e154e5c1 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:10:04.055574 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f2b9b1-a544-4c23-ba4e-01bdd12a6ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:04.056837 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f2b9b1-a544-4c23-ba4e-01bdd12a6ee0 None admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:04.057337 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f2b9b1-a544-4c23-ba4e-01bdd12a6ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:04.057806 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f2b9b1-a544-4c23-ba4e-01bdd12a6ee0 None admin] PATH_INFO: `/v3/projects/625d29bdd4ad47ee92370dc3404fe414/users/da56a9ca0e2e4d098b4bc6cd218c694f/roles/60af5088793e4379b88821a86552a344` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:04.065207 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f2b9b1-a544-4c23-ba4e-01bdd12a6ee0 None admin] RBAC: Authorizing `identity:create_grant(project_id=625d29bdd4ad47ee92370dc3404fe414, user_id=da56a9ca0e2e4d098b4bc6cd218c694f, role_id=60af5088793e4379b88821a86552a344)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:04.067024 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f2b9b1-a544-4c23-ba4e-01bdd12a6ee0 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:04.145298 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 111/234] 10.222.0.58 () {58 vars in 1439 bytes} [Tue May 17 19:10:03 2022] PUT /identity/v3/projects/625d29bdd4ad47ee92370dc3404fe414/users/da56a9ca0e2e4d098b4bc6cd218c694f/roles/60af5088793e4379b88821a86552a344 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:10:05.181610 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f1aa3c-a6dc-4e98-93a0-35086ff1e4bc None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:05.182110 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f1aa3c-a6dc-4e98-93a0-35086ff1e4bc None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:05.182183 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f1aa3c-a6dc-4e98-93a0-35086ff1e4bc None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:05.183552 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 124/235] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:10:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:10:05.189087 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e823b7d-f188-4cb7-9e66-8dad2846bd4a None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:05.189528 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e823b7d-f188-4cb7-9e66-8dad2846bd4a None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:05.189718 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e823b7d-f188-4cb7-9e66-8dad2846bd4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:05.212880 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-8e823b7d-f188-4cb7-9e66-8dad2846bd4a None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:10:05.214030 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-8e823b7d-f188-4cb7-9e66-8dad2846bd4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:10:05.279840 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b315948e-0c68-457b-9c1a-2da6446b0182 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:05.280464 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b315948e-0c68-457b-9c1a-2da6446b0182 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:05.280929 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b315948e-0c68-457b-9c1a-2da6446b0182 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:05.282750 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 125/236] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:10:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:10:05.288145 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa14f06f-8c85-4cfd-a952-96a8dd9cdaf1 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:05.288646 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa14f06f-8c85-4cfd-a952-96a8dd9cdaf1 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:05.289135 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa14f06f-8c85-4cfd-a952-96a8dd9cdaf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:05.297903 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 112/237] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:10:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 17 19:10:05.308593 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-aa14f06f-8c85-4cfd-a952-96a8dd9cdaf1 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:10:05.309993 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-aa14f06f-8c85-4cfd-a952-96a8dd9cdaf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:10:05.338264 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e823b7d-f188-4cb7-9e66-8dad2846bd4a None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:05.401668 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 126/238] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:10:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 17 19:10:05.432775 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa14f06f-8c85-4cfd-a952-96a8dd9cdaf1 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:05.437813 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e823b7d-f188-4cb7-9e66-8dad2846bd4a None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:05.485631 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3953ad16-c72e-4034-91d7-8976ad55d601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:05.486354 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3953ad16-c72e-4034-91d7-8976ad55d601 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:05.486636 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3953ad16-c72e-4034-91d7-8976ad55d601 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:05.486908 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3953ad16-c72e-4034-91d7-8976ad55d601 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:05.492368 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3953ad16-c72e-4034-91d7-8976ad55d601 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:05.493837 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3953ad16-c72e-4034-91d7-8976ad55d601 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:05.495831 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 113/239] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:10:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:10:05.500911 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3953ad16-c72e-4034-91d7-8976ad55d601 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:05.550943 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa14f06f-8c85-4cfd-a952-96a8dd9cdaf1 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:05.588565 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d35a111-d864-46bd-bbb6-0defcbba37e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:05.589746 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d35a111-d864-46bd-bbb6-0defcbba37e7 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:05.590214 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d35a111-d864-46bd-bbb6-0defcbba37e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:05.590804 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d35a111-d864-46bd-bbb6-0defcbba37e7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:05.601678 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d35a111-d864-46bd-bbb6-0defcbba37e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 17 19:10:05.609478 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-8d35a111-d864-46bd-bbb6-0defcbba37e7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 17 19:10:05.609478 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:05.609478 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:05.609478 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:05.609478 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:05.609478 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:05.609478 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:05.609478 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:05.609478 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:05.609478 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:05.609478 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:05.609478 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:05.609478 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 17 19:10:05.609478 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) May 17 19:10:05.609478 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 17 19:10:05.609478 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 17 19:10:05.610349 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 17 19:10:05.610349 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 17 19:10:05.610349 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 17 19:10:05.610349 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 17 19:10:05.610349 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:05.610349 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:05.610349 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:05.610349 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:05.610349 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:05.610349 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:05.610349 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:05.610349 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:10:05.610349 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:05.610349 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:05.610349 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:05.610349 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:05.611119 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:05.611119 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:05.611119 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:05.611119 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:05.611119 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 17 19:10:05.611119 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 17 19:10:05.611119 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 17 19:10:05.611119 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 17 19:10:05.611119 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 17 19:10:05.611119 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 17 19:10:05.611119 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 17 19:10:05.611119 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:10:05.613518 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 127/240] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 17 19:10:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 17 19:10:05.618752 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3953ad16-c72e-4034-91d7-8976ad55d601 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:05.619957 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d35a111-d864-46bd-bbb6-0defcbba37e7 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:05.673034 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee395c57-036d-4760-ba69-f4d34d43b80d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:05.673917 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee395c57-036d-4760-ba69-f4d34d43b80d None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:05.674026 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee395c57-036d-4760-ba69-f4d34d43b80d None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:05.674257 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee395c57-036d-4760-ba69-f4d34d43b80d None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:05.674974 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee395c57-036d-4760-ba69-f4d34d43b80d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:05.676964 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee395c57-036d-4760-ba69-f4d34d43b80d None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:05.734379 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d35a111-d864-46bd-bbb6-0defcbba37e7 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:05.769126 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1d010e-cf96-4f21-ad2f-451a2639b656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:05.770111 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1d010e-cf96-4f21-ad2f-451a2639b656 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:05.770580 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1d010e-cf96-4f21-ad2f-451a2639b656 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:05.771059 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1d010e-cf96-4f21-ad2f-451a2639b656 None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:05.772040 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1d010e-cf96-4f21-ad2f-451a2639b656 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:05.772607 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1d010e-cf96-4f21-ad2f-451a2639b656 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:05.774355 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1d010e-cf96-4f21-ad2f-451a2639b656 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:05.785077 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 128/241] 10.222.0.58 () {56 vars in 1101 bytes} [Tue May 17 19:10:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:05.791349 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1d010e-cf96-4f21-ad2f-451a2639b656 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:05.791957 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 114/242] 10.222.0.58 () {60 vars in 1132 bytes} [Tue May 17 19:10:05 2022] POST /identity/v3/users => generated 293 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 17 19:10:05.923479 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1d010e-cf96-4f21-ad2f-451a2639b656 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:05.980715 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b95925-43ff-4b4b-8ebd-0c219d87aee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:05.982009 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b95925-43ff-4b4b-8ebd-0c219d87aee6 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:05.982485 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b95925-43ff-4b4b-8ebd-0c219d87aee6 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:05.982948 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b95925-43ff-4b4b-8ebd-0c219d87aee6 None admin] PATH_INFO: `/v3/users/9132c66a30d74f2c9d97e6443f313eea` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:05.998702 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b95925-43ff-4b4b-8ebd-0c219d87aee6 None admin] RBAC: Authorizing `identity:get_user(user_id=9132c66a30d74f2c9d97e6443f313eea)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:06.000688 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b95925-43ff-4b4b-8ebd-0c219d87aee6 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:06.002951 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 129/243] 10.222.0.58 () {56 vars in 1177 bytes} [Tue May 17 19:10:05 2022] GET /identity/v3/users/9132c66a30d74f2c9d97e6443f313eea => generated 297 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:10:06.008833 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee395c57-036d-4760-ba69-f4d34d43b80d None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:06.125382 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee395c57-036d-4760-ba69-f4d34d43b80d None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:06.159318 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ae8ffc-716d-445f-a563-211e9a2baa58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:06.160001 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ae8ffc-716d-445f-a563-211e9a2baa58 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:06.160286 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ae8ffc-716d-445f-a563-211e9a2baa58 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:06.160560 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ae8ffc-716d-445f-a563-211e9a2baa58 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:06.229563 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ae8ffc-716d-445f-a563-211e9a2baa58 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:06.231075 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ae8ffc-716d-445f-a563-211e9a2baa58 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:06.332538 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 115/244] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 17 19:10:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 17 19:10:06.338108 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b95925-43ff-4b4b-8ebd-0c219d87aee6 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:06.445030 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b95925-43ff-4b4b-8ebd-0c219d87aee6 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:06.479704 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70276204-a78d-4402-bf3c-4f3a8eafcae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:06.480569 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70276204-a78d-4402-bf3c-4f3a8eafcae6 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:06.480851 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70276204-a78d-4402-bf3c-4f3a8eafcae6 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:06.481126 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70276204-a78d-4402-bf3c-4f3a8eafcae6 None admin] PATH_INFO: `/v3/projects/625d29bdd4ad47ee92370dc3404fe414` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:06.482325 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70276204-a78d-4402-bf3c-4f3a8eafcae6 None admin] RBAC: Authorizing `identity:get_project(project_id=625d29bdd4ad47ee92370dc3404fe414)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:06.483816 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70276204-a78d-4402-bf3c-4f3a8eafcae6 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:06.485825 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 130/245] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 17 19:10:06 2022] GET /identity/v3/projects/625d29bdd4ad47ee92370dc3404fe414 => generated 302 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:10:06.491887 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ae8ffc-716d-445f-a563-211e9a2baa58 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:06.606513 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be62e75-c632-43de-accb-352ce287be50 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:06.606849 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be62e75-c632-43de-accb-352ce287be50 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:06.607059 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be62e75-c632-43de-accb-352ce287be50 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:06.608491 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ae8ffc-716d-445f-a563-211e9a2baa58 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:06.608726 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 131/246] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:10:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:10:06.616010 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30df7f62-a402-4608-84c7-7c71e259267c None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:06.616888 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30df7f62-a402-4608-84c7-7c71e259267c None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:06.617137 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30df7f62-a402-4608-84c7-7c71e259267c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:06.649299 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-30df7f62-a402-4608-84c7-7c71e259267c None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:10:06.650626 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-30df7f62-a402-4608-84c7-7c71e259267c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:10:06.658282 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ecd25b-8327-40d6-a199-e9aebbb86939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:06.659159 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ecd25b-8327-40d6-a199-e9aebbb86939 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:06.659614 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ecd25b-8327-40d6-a199-e9aebbb86939 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:06.660075 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ecd25b-8327-40d6-a199-e9aebbb86939 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:06.661055 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ecd25b-8327-40d6-a199-e9aebbb86939 None admin] RBAC: Adding query filter params (role.id=8e3500b266214ae2be189641a1b4dbf1, scope.project.id=625d29bdd4ad47ee92370dc3404fe414, user.id=9132c66a30d74f2c9d97e6443f313eea) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:06.661507 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ecd25b-8327-40d6-a199-e9aebbb86939 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:06.664255 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ecd25b-8327-40d6-a199-e9aebbb86939 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:06.673454 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 116/247] 10.222.0.58 () {56 vars in 1374 bytes} [Tue May 17 19:10:06 2022] GET /identity/v3/role_assignments?user.id=9132c66a30d74f2c9d97e6443f313eea&scope.project.id=625d29bdd4ad47ee92370dc3404fe414&role.id=8e3500b266214ae2be189641a1b4dbf1 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:06.739966 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 132/248] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:10:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 17 19:10:06.764499 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ecd25b-8327-40d6-a199-e9aebbb86939 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:06.806097 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80facbd-bad8-4642-a2b2-02862b6c5a26 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:06.806386 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80facbd-bad8-4642-a2b2-02862b6c5a26 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:06.806682 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80facbd-bad8-4642-a2b2-02862b6c5a26 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:06.808371 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 133/249] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:10:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 17 19:10:06.819217 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f00dd0-57ae-4801-9e22-dc6c513d8044 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:06.819701 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f00dd0-57ae-4801-9e22-dc6c513d8044 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:06.820071 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f00dd0-57ae-4801-9e22-dc6c513d8044 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:06.840488 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-77f00dd0-57ae-4801-9e22-dc6c513d8044 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:10:06.841833 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-77f00dd0-57ae-4801-9e22-dc6c513d8044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:10:06.889143 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ecd25b-8327-40d6-a199-e9aebbb86939 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:06.929175 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92be222-b0d2-4b32-9a98-c5d04e640fca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:06.929975 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92be222-b0d2-4b32-9a98-c5d04e640fca None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:06.930247 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92be222-b0d2-4b32-9a98-c5d04e640fca None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:06.930573 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92be222-b0d2-4b32-9a98-c5d04e640fca None admin] PATH_INFO: `/v3/roles/member` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:06.951519 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92be222-b0d2-4b32-9a98-c5d04e640fca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 17 19:10:06.957637 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-e92be222-b0d2-4b32-9a98-c5d04e640fca None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 17 19:10:06.957637 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:06.957637 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:06.957637 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:06.957637 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:06.957637 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:06.957637 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:06.957637 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:06.957637 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:06.957637 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:06.957637 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:06.957637 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:06.957637 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 17 19:10:06.957637 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) May 17 19:10:06.957637 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 17 19:10:06.957637 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 17 19:10:06.958522 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 17 19:10:06.958522 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 17 19:10:06.958522 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 17 19:10:06.958522 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 17 19:10:06.958522 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:06.958522 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:06.958522 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:06.958522 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:06.958522 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:06.958522 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:06.958522 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:06.958522 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:10:06.958522 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:06.958522 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:06.958522 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:06.958522 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:06.959329 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:06.959329 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:06.959329 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:06.959329 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:06.959329 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 17 19:10:06.959329 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 17 19:10:06.959329 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 17 19:10:06.959329 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 17 19:10:06.959329 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 17 19:10:06.959329 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 17 19:10:06.959329 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 17 19:10:06.959329 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:10:06.959329 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 117/250] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 17 19:10:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:10:06.967036 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92be222-b0d2-4b32-9a98-c5d04e640fca None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:06.989213 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 134/251] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:10:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 17 19:10:06.995909 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12c9832-1c5b-4bb7-aab6-2974377e5454 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:06.996120 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12c9832-1c5b-4bb7-aab6-2974377e5454 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:06.996387 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12c9832-1c5b-4bb7-aab6-2974377e5454 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:06.997823 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 135/252] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:10:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 17 19:10:07.003058 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32e729a-0648-4fe6-a917-b4c8a9134c6f None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:07.003647 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32e729a-0648-4fe6-a917-b4c8a9134c6f None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:07.004115 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32e729a-0648-4fe6-a917-b4c8a9134c6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:07.020919 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-c32e729a-0648-4fe6-a917-b4c8a9134c6f None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:10:07.022905 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-c32e729a-0648-4fe6-a917-b4c8a9134c6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:10:07.080319 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92be222-b0d2-4b32-9a98-c5d04e640fca None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:07.090834 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 136/253] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:10:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 17 19:10:07.096161 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32e729a-0648-4fe6-a917-b4c8a9134c6f None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:07.126609 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75737175-e535-422b-b787-f40f6a517b2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:07.127283 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75737175-e535-422b-b787-f40f6a517b2c None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:07.127587 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75737175-e535-422b-b787-f40f6a517b2c None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:07.127864 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75737175-e535-422b-b787-f40f6a517b2c None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:07.128610 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75737175-e535-422b-b787-f40f6a517b2c None admin] RBAC: Adding query filter params (name=member) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:07.128915 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75737175-e535-422b-b787-f40f6a517b2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:07.131615 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75737175-e535-422b-b787-f40f6a517b2c None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:07.142356 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 118/254] 10.222.0.58 () {56 vars in 1101 bytes} [Tue May 17 19:10:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:07.143274 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75737175-e535-422b-b787-f40f6a517b2c None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:07.255152 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75737175-e535-422b-b787-f40f6a517b2c None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:07.299721 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd90dfba-f6cb-472f-8be3-af17ed3eac62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:07.300526 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd90dfba-f6cb-472f-8be3-af17ed3eac62 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:07.300806 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd90dfba-f6cb-472f-8be3-af17ed3eac62 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:07.301108 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd90dfba-f6cb-472f-8be3-af17ed3eac62 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:07.323694 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32e729a-0648-4fe6-a917-b4c8a9134c6f None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:07.326050 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd90dfba-f6cb-472f-8be3-af17ed3eac62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 17 19:10:07.334737 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-dd90dfba-f6cb-472f-8be3-af17ed3eac62 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 17 19:10:07.334737 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:07.334737 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:07.334737 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:07.334737 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:07.334737 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:07.334737 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:07.334737 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:07.334737 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:07.334737 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:07.334737 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:07.334737 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:07.334737 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 17 19:10:07.334737 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) May 17 19:10:07.334737 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 17 19:10:07.334737 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 17 19:10:07.336215 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 17 19:10:07.336215 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 17 19:10:07.336215 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 17 19:10:07.336215 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 17 19:10:07.336215 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:07.336215 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:07.336215 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:07.336215 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:07.336215 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:07.336215 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:07.336215 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:07.336215 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:10:07.336215 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:07.336215 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:07.336215 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:07.336215 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:07.338443 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:07.338443 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:07.338443 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:07.338443 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:07.338443 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 17 19:10:07.338443 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 17 19:10:07.338443 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 17 19:10:07.338443 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 17 19:10:07.338443 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 17 19:10:07.338443 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 17 19:10:07.338443 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 17 19:10:07.338443 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:10:07.338443 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 119/255] 10.222.0.58 () {56 vars in 1096 bytes} [Tue May 17 19:10:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:10:07.338443 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd90dfba-f6cb-472f-8be3-af17ed3eac62 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:07.424755 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff41107-ec7a-4c22-9c6f-a533f7cfb8af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:07.425696 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff41107-ec7a-4c22-9c6f-a533f7cfb8af None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:07.425882 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff41107-ec7a-4c22-9c6f-a533f7cfb8af None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:07.426152 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff41107-ec7a-4c22-9c6f-a533f7cfb8af None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:07.452876 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd90dfba-f6cb-472f-8be3-af17ed3eac62 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:07.460972 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff41107-ec7a-4c22-9c6f-a533f7cfb8af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 17 19:10:07.477186 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-0ff41107-ec7a-4c22-9c6f-a533f7cfb8af None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 17 19:10:07.477186 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:07.477186 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:07.477186 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:07.477186 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:07.477186 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:07.477186 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:07.477186 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:07.477186 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:07.477186 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:07.477186 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:07.477186 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:07.477186 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 17 19:10:07.477186 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) May 17 19:10:07.477186 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 17 19:10:07.477186 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 17 19:10:07.479615 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 17 19:10:07.479615 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 17 19:10:07.479615 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 17 19:10:07.479615 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 17 19:10:07.479615 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:07.479615 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:07.479615 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:07.479615 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:07.479615 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:07.479615 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:07.479615 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:07.479615 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:10:07.479615 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:07.479615 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:07.479615 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:07.479615 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:07.481617 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:07.481617 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:07.481617 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:07.481617 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:07.481617 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 17 19:10:07.481617 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 17 19:10:07.481617 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 17 19:10:07.481617 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 17 19:10:07.481617 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 17 19:10:07.481617 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 17 19:10:07.481617 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 17 19:10:07.481617 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:10:07.481617 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 137/256] 10.222.0.58 () {56 vars in 1114 bytes} [Tue May 17 19:10:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 387 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:10:07.481617 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff41107-ec7a-4c22-9c6f-a533f7cfb8af None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:07.498163 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd1d733-b06b-4962-86a3-f68f4c22fa4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:07.499033 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd1d733-b06b-4962-86a3-f68f4c22fa4f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:07.499310 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd1d733-b06b-4962-86a3-f68f4c22fa4f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:07.499622 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd1d733-b06b-4962-86a3-f68f4c22fa4f None admin] PATH_INFO: `/v3/users/614688dc21d94bc7a5f85a06df453136` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:07.502123 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd1d733-b06b-4962-86a3-f68f4c22fa4f None admin] RBAC: Authorizing `identity:get_user(user_id=614688dc21d94bc7a5f85a06df453136)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:07.503624 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd1d733-b06b-4962-86a3-f68f4c22fa4f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:07.505462 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 120/257] 10.222.0.58 () {56 vars in 1177 bytes} [Tue May 17 19:10:07 2022] GET /identity/v3/users/614688dc21d94bc7a5f85a06df453136 => generated 297 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:07.505996 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd1d733-b06b-4962-86a3-f68f4c22fa4f None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:07.671246 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff41107-ec7a-4c22-9c6f-a533f7cfb8af None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:07.698029 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd1d733-b06b-4962-86a3-f68f4c22fa4f None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:07.737523 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bd6771-2e5f-4aff-b415-7f53e669f5b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:07.738191 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bd6771-2e5f-4aff-b415-7f53e669f5b5 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:07.738472 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bd6771-2e5f-4aff-b415-7f53e669f5b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:07.738746 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bd6771-2e5f-4aff-b415-7f53e669f5b5 None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:07.739506 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bd6771-2e5f-4aff-b415-7f53e669f5b5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:07.739814 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bd6771-2e5f-4aff-b415-7f53e669f5b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:07.740253 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1513aa-d065-4553-8f81-36851eda89d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:07.741660 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1513aa-d065-4553-8f81-36851eda89d6 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:07.741761 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bd6771-2e5f-4aff-b415-7f53e669f5b5 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:07.742217 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1513aa-d065-4553-8f81-36851eda89d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:07.742736 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1513aa-d065-4553-8f81-36851eda89d6 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:07.743813 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1513aa-d065-4553-8f81-36851eda89d6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:07.744314 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1513aa-d065-4553-8f81-36851eda89d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:07.745855 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1513aa-d065-4553-8f81-36851eda89d6 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:07.754210 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 121/258] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 17 19:10:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:07.755693 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1513aa-d065-4553-8f81-36851eda89d6 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:07.762181 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 138/259] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 17 19:10:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:07.763047 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bd6771-2e5f-4aff-b415-7f53e669f5b5 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:07.903618 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1513aa-d065-4553-8f81-36851eda89d6 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:07.946944 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4775d96c-9ec2-4407-a1eb-dd75ade5c2ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:07.947833 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4775d96c-9ec2-4407-a1eb-dd75ade5c2ad None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:07.948110 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4775d96c-9ec2-4407-a1eb-dd75ade5c2ad None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:07.948377 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4775d96c-9ec2-4407-a1eb-dd75ade5c2ad None admin] PATH_INFO: `/v3/projects/625d29bdd4ad47ee92370dc3404fe414` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:07.956292 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4775d96c-9ec2-4407-a1eb-dd75ade5c2ad None admin] RBAC: Authorizing `identity:get_project(project_id=625d29bdd4ad47ee92370dc3404fe414)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:07.958344 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4775d96c-9ec2-4407-a1eb-dd75ade5c2ad None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:07.959964 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bd6771-2e5f-4aff-b415-7f53e669f5b5 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:07.961828 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 122/260] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 17 19:10:07 2022] GET /identity/v3/projects/625d29bdd4ad47ee92370dc3404fe414 => generated 302 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:07.971519 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4775d96c-9ec2-4407-a1eb-dd75ade5c2ad None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:08.002524 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9ab246-4ee9-4b15-ab09-7ecd0d562595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:08.003824 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9ab246-4ee9-4b15-ab09-7ecd0d562595 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:08.004386 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9ab246-4ee9-4b15-ab09-7ecd0d562595 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:08.004859 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9ab246-4ee9-4b15-ab09-7ecd0d562595 None admin] PATH_INFO: `/v3/users/da56a9ca0e2e4d098b4bc6cd218c694f` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:08.006269 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9ab246-4ee9-4b15-ab09-7ecd0d562595 None admin] RBAC: Authorizing `identity:get_user(user_id=da56a9ca0e2e4d098b4bc6cd218c694f)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:08.008612 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9ab246-4ee9-4b15-ab09-7ecd0d562595 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:08.010511 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 139/261] 10.222.0.58 () {56 vars in 1177 bytes} [Tue May 17 19:10:07 2022] GET /identity/v3/users/da56a9ca0e2e4d098b4bc6cd218c694f => generated 283 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:08.011431 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9ab246-4ee9-4b15-ab09-7ecd0d562595 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:08.113038 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4775d96c-9ec2-4407-a1eb-dd75ade5c2ad None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:08.127772 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9ab246-4ee9-4b15-ab09-7ecd0d562595 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:08.147160 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33813949-482d-4ffe-9b8c-6d15fff2d3be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:08.148658 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33813949-482d-4ffe-9b8c-6d15fff2d3be None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:08.148946 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33813949-482d-4ffe-9b8c-6d15fff2d3be None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:08.149216 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33813949-482d-4ffe-9b8c-6d15fff2d3be None admin] PATH_INFO: `/v3/users/da56a9ca0e2e4d098b4bc6cd218c694f` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:08.150776 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33813949-482d-4ffe-9b8c-6d15fff2d3be None admin] RBAC: Authorizing `identity:get_user(user_id=da56a9ca0e2e4d098b4bc6cd218c694f)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:08.151906 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33813949-482d-4ffe-9b8c-6d15fff2d3be None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:08.153546 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 123/262] 10.222.0.58 () {56 vars in 1177 bytes} [Tue May 17 19:10:07 2022] GET /identity/v3/users/da56a9ca0e2e4d098b4bc6cd218c694f => generated 283 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:08.154615 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33813949-482d-4ffe-9b8c-6d15fff2d3be None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:08.165070 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e1cafc-41de-4a57-aecf-b47f30fa8705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:08.166281 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e1cafc-41de-4a57-aecf-b47f30fa8705 None admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:08.166773 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e1cafc-41de-4a57-aecf-b47f30fa8705 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:08.167234 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e1cafc-41de-4a57-aecf-b47f30fa8705 None admin] PATH_INFO: `/v3/projects/625d29bdd4ad47ee92370dc3404fe414/users/614688dc21d94bc7a5f85a06df453136/roles/114e6a16e63447d1924707ace5aace25` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:08.169402 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e1cafc-41de-4a57-aecf-b47f30fa8705 None admin] RBAC: Authorizing `identity:create_grant(project_id=625d29bdd4ad47ee92370dc3404fe414, user_id=614688dc21d94bc7a5f85a06df453136, role_id=114e6a16e63447d1924707ace5aace25)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:08.171024 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e1cafc-41de-4a57-aecf-b47f30fa8705 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:08.253839 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33813949-482d-4ffe-9b8c-6d15fff2d3be None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:08.279593 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 140/263] 10.222.0.58 () {58 vars in 1439 bytes} [Tue May 17 19:10:08 2022] PUT /identity/v3/projects/625d29bdd4ad47ee92370dc3404fe414/users/614688dc21d94bc7a5f85a06df453136/roles/114e6a16e63447d1924707ace5aace25 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:10:08.281833 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e1cafc-41de-4a57-aecf-b47f30fa8705 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:08.307885 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01906749-04ab-41ef-ad0a-d96cf9717208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:08.307885 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01906749-04ab-41ef-ad0a-d96cf9717208 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:08.307885 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01906749-04ab-41ef-ad0a-d96cf9717208 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:08.307885 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01906749-04ab-41ef-ad0a-d96cf9717208 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:08.424359 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01906749-04ab-41ef-ad0a-d96cf9717208 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:08.425870 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01906749-04ab-41ef-ad0a-d96cf9717208 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:08.547437 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e1cafc-41de-4a57-aecf-b47f30fa8705 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:08.571648 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 124/264] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 17 19:10:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 17 19:10:08.576972 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01906749-04ab-41ef-ad0a-d96cf9717208 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:08.640209 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79739095-2926-42d9-a7bd-0b9a28e9e6b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:08.640876 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79739095-2926-42d9-a7bd-0b9a28e9e6b3 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:08.641164 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79739095-2926-42d9-a7bd-0b9a28e9e6b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:08.641446 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79739095-2926-42d9-a7bd-0b9a28e9e6b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:08.699049 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01906749-04ab-41ef-ad0a-d96cf9717208 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:08.733966 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f65d13-a3c4-4893-8e7f-b4cd8244111e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:08.734885 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f65d13-a3c4-4893-8e7f-b4cd8244111e None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:08.735162 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f65d13-a3c4-4893-8e7f-b4cd8244111e None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:08.735475 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f65d13-a3c4-4893-8e7f-b4cd8244111e None admin] PATH_INFO: `/v3/projects/625d29bdd4ad47ee92370dc3404fe414` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:08.736688 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f65d13-a3c4-4893-8e7f-b4cd8244111e None admin] RBAC: Authorizing `identity:get_project(project_id=625d29bdd4ad47ee92370dc3404fe414)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:08.738149 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f65d13-a3c4-4893-8e7f-b4cd8244111e None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:08.740184 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 125/265] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 17 19:10:08 2022] GET /identity/v3/projects/625d29bdd4ad47ee92370dc3404fe414 => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:10:08.746392 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f65d13-a3c4-4893-8e7f-b4cd8244111e None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:08.754948 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79739095-2926-42d9-a7bd-0b9a28e9e6b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:08.756478 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79739095-2926-42d9-a7bd-0b9a28e9e6b3 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:08.846038 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f65d13-a3c4-4893-8e7f-b4cd8244111e None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:08.860550 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 141/266] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 17 19:10:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 576 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 17 19:10:08.867556 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79739095-2926-42d9-a7bd-0b9a28e9e6b3 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:08.881778 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9389f682-0810-4e04-b911-39e225cae99a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:08.882433 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9389f682-0810-4e04-b911-39e225cae99a None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:08.882708 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9389f682-0810-4e04-b911-39e225cae99a None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:08.882984 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9389f682-0810-4e04-b911-39e225cae99a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:08.883850 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9389f682-0810-4e04-b911-39e225cae99a None admin] RBAC: Adding query filter params (role.id=1055a3235b0143e58f1090c1e154e5c1, scope.project.id=625d29bdd4ad47ee92370dc3404fe414, user.id=da56a9ca0e2e4d098b4bc6cd218c694f) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:08.884156 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9389f682-0810-4e04-b911-39e225cae99a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:08.885606 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9389f682-0810-4e04-b911-39e225cae99a None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:08.891716 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 126/267] 10.222.0.58 () {56 vars in 1374 bytes} [Tue May 17 19:10:08 2022] GET /identity/v3/role_assignments?user.id=da56a9ca0e2e4d098b4bc6cd218c694f&scope.project.id=625d29bdd4ad47ee92370dc3404fe414&role.id=1055a3235b0143e58f1090c1e154e5c1 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:10:08.970245 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79739095-2926-42d9-a7bd-0b9a28e9e6b3 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:09.015363 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9596bc9-9e58-4568-b1b3-e269940be42e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:09.016288 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9596bc9-9e58-4568-b1b3-e269940be42e None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:09.016572 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9596bc9-9e58-4568-b1b3-e269940be42e None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:09.016845 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9596bc9-9e58-4568-b1b3-e269940be42e None admin] PATH_INFO: `/v3/projects/625d29bdd4ad47ee92370dc3404fe414` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:09.018107 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9596bc9-9e58-4568-b1b3-e269940be42e None admin] RBAC: Authorizing `identity:get_project(project_id=625d29bdd4ad47ee92370dc3404fe414)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:09.019667 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9596bc9-9e58-4568-b1b3-e269940be42e None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:09.021445 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 142/268] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 17 19:10:08 2022] GET /identity/v3/projects/625d29bdd4ad47ee92370dc3404fe414 => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:10:09.027400 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9389f682-0810-4e04-b911-39e225cae99a None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:09.149668 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9389f682-0810-4e04-b911-39e225cae99a None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:09.184245 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa926b7-ae9d-4105-9a3c-d7dcba3d97ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:09.184968 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa926b7-ae9d-4105-9a3c-d7dcba3d97ed None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:09.185238 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa926b7-ae9d-4105-9a3c-d7dcba3d97ed None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:09.185510 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa926b7-ae9d-4105-9a3c-d7dcba3d97ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:09.186382 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa926b7-ae9d-4105-9a3c-d7dcba3d97ed None admin] RBAC: Adding query filter params (role.id=60af5088793e4379b88821a86552a344, scope.project.id=625d29bdd4ad47ee92370dc3404fe414, user.id=da56a9ca0e2e4d098b4bc6cd218c694f) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:09.186692 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa926b7-ae9d-4105-9a3c-d7dcba3d97ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:09.188187 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa926b7-ae9d-4105-9a3c-d7dcba3d97ed None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:09.194779 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 127/269] 10.222.0.58 () {56 vars in 1374 bytes} [Tue May 17 19:10:09 2022] GET /identity/v3/role_assignments?user.id=da56a9ca0e2e4d098b4bc6cd218c694f&scope.project.id=625d29bdd4ad47ee92370dc3404fe414&role.id=60af5088793e4379b88821a86552a344 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:09.466834 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe05aa1-0fac-4a47-b1d2-4b81d9e6ec22 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:09.467352 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe05aa1-0fac-4a47-b1d2-4b81d9e6ec22 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:09.467752 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe05aa1-0fac-4a47-b1d2-4b81d9e6ec22 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:09.469552 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 143/270] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:10:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:10:09.476028 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748b314f-952c-4a2e-a45e-a60c4ab56b24 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:09.476522 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748b314f-952c-4a2e-a45e-a60c4ab56b24 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:09.476593 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748b314f-952c-4a2e-a45e-a60c4ab56b24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:09.496567 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-748b314f-952c-4a2e-a45e-a60c4ab56b24 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:10:09.497790 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-748b314f-952c-4a2e-a45e-a60c4ab56b24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:10:09.556435 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9468f6d-8c69-40dc-9570-bc66ef67716c None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:09.556946 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9468f6d-8c69-40dc-9570-bc66ef67716c None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:09.557019 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9468f6d-8c69-40dc-9570-bc66ef67716c None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:09.559080 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 144/271] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:10:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:10:09.567782 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550c4e62-efcc-47cf-9b7d-dae5cf4329c0 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:09.568106 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550c4e62-efcc-47cf-9b7d-dae5cf4329c0 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:09.568337 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550c4e62-efcc-47cf-9b7d-dae5cf4329c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:09.590776 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 128/272] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:10:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 17 19:10:09.597319 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-550c4e62-efcc-47cf-9b7d-dae5cf4329c0 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:10:09.598561 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-550c4e62-efcc-47cf-9b7d-dae5cf4329c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:10:09.615097 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748b314f-952c-4a2e-a45e-a60c4ab56b24 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:09.718646 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 145/273] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:10:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 17 19:10:09.747219 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550c4e62-efcc-47cf-9b7d-dae5cf4329c0 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:09.748900 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748b314f-952c-4a2e-a45e-a60c4ab56b24 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:09.787356 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a3f4e8-1f0e-4065-831e-fb22503ad928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:09.788232 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a3f4e8-1f0e-4065-831e-fb22503ad928 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:09.788510 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a3f4e8-1f0e-4065-831e-fb22503ad928 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:09.788780 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a3f4e8-1f0e-4065-831e-fb22503ad928 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:09.801072 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a3f4e8-1f0e-4065-831e-fb22503ad928 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 17 19:10:09.811045 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-c3a3f4e8-1f0e-4065-831e-fb22503ad928 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 17 19:10:09.811045 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:09.811045 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:09.811045 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:09.811045 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:09.811045 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:09.811045 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:09.811045 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:09.811045 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:09.811045 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:09.811045 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:09.811045 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:09.811045 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 17 19:10:09.811045 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) May 17 19:10:09.811045 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 17 19:10:09.811045 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 17 19:10:09.811964 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 17 19:10:09.811964 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 17 19:10:09.811964 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 17 19:10:09.811964 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 17 19:10:09.811964 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:09.811964 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:09.811964 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:09.811964 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:09.811964 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:09.811964 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:09.811964 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:09.811964 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:10:09.811964 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:09.811964 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:09.811964 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:09.811964 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:09.812772 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:09.812772 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:09.812772 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:09.812772 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:09.812772 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 17 19:10:09.812772 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 17 19:10:09.812772 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 17 19:10:09.812772 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 17 19:10:09.812772 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 17 19:10:09.812772 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 17 19:10:09.812772 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 17 19:10:09.812772 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:10:09.812772 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 129/274] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 17 19:10:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 17 19:10:09.818736 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dc7fa4-a3b7-4370-8ae5-3e5418c02d52 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:09.818736 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dc7fa4-a3b7-4370-8ae5-3e5418c02d52 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:09.818736 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dc7fa4-a3b7-4370-8ae5-3e5418c02d52 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:09.818736 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 130/275] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:10:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:10:09.821010 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dc7fa4-a3b7-4370-8ae5-3e5418c02d52 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:09.853294 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550c4e62-efcc-47cf-9b7d-dae5cf4329c0 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:09.887875 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46d8a5f-ffb9-43cb-a9ae-55354c47dfd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:09.888725 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46d8a5f-ffb9-43cb-a9ae-55354c47dfd4 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:09.889029 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46d8a5f-ffb9-43cb-a9ae-55354c47dfd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:09.889314 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46d8a5f-ffb9-43cb-a9ae-55354c47dfd4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:09.890521 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46d8a5f-ffb9-43cb-a9ae-55354c47dfd4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:09.892072 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46d8a5f-ffb9-43cb-a9ae-55354c47dfd4 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:09.893721 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 146/276] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:10:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:09.893721 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8285ce34-e631-4821-a307-989b0f3e591d None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:09.893721 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8285ce34-e631-4821-a307-989b0f3e591d None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:09.893721 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8285ce34-e631-4821-a307-989b0f3e591d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:09.915290 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-8285ce34-e631-4821-a307-989b0f3e591d None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:10:09.916526 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-8285ce34-e631-4821-a307-989b0f3e591d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:10:09.926235 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dc7fa4-a3b7-4370-8ae5-3e5418c02d52 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:09.963468 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c92d650-8ec1-444a-9709-d9c86f5f7722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:09.964172 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c92d650-8ec1-444a-9709-d9c86f5f7722 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:09.964445 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c92d650-8ec1-444a-9709-d9c86f5f7722 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:09.964705 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c92d650-8ec1-444a-9709-d9c86f5f7722 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:09.965443 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c92d650-8ec1-444a-9709-d9c86f5f7722 None admin] RBAC: Adding query filter params (name=member) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:09.965751 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c92d650-8ec1-444a-9709-d9c86f5f7722 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:09.967125 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c92d650-8ec1-444a-9709-d9c86f5f7722 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:09.980496 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 131/277] 10.222.0.58 () {56 vars in 1101 bytes} [Tue May 17 19:10:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:09.980496 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c92d650-8ec1-444a-9709-d9c86f5f7722 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:09.992196 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 147/278] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:10:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 17 19:10:09.993042 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8285ce34-e631-4821-a307-989b0f3e591d None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:10.089487 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c92d650-8ec1-444a-9709-d9c86f5f7722 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:10.090713 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8285ce34-e631-4821-a307-989b0f3e591d None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:10.123167 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c06d371-93f3-4f26-a3c1-19c87c3dc75a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:10.123892 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c06d371-93f3-4f26-a3c1-19c87c3dc75a None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:10.124169 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c06d371-93f3-4f26-a3c1-19c87c3dc75a None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:10.124327 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05df287-afbc-45bb-a640-574061900e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:10.124453 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c06d371-93f3-4f26-a3c1-19c87c3dc75a None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:10.125166 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05df287-afbc-45bb-a640-574061900e2b None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:10.125261 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c06d371-93f3-4f26-a3c1-19c87c3dc75a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:10.125493 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05df287-afbc-45bb-a640-574061900e2b None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:10.125766 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05df287-afbc-45bb-a640-574061900e2b None admin] PATH_INFO: `/v3/users/4a8932999b1f4466a222b252ca4ddb53` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:10.126730 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c06d371-93f3-4f26-a3c1-19c87c3dc75a None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:10.139221 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05df287-afbc-45bb-a640-574061900e2b None admin] RBAC: Authorizing `identity:get_user(user_id=4a8932999b1f4466a222b252ca4ddb53)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:10.140696 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05df287-afbc-45bb-a640-574061900e2b None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:10.142561 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 148/279] 10.222.0.58 () {56 vars in 1177 bytes} [Tue May 17 19:10:09 2022] GET /identity/v3/users/4a8932999b1f4466a222b252ca4ddb53 => generated 293 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:10.143129 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05df287-afbc-45bb-a640-574061900e2b None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:10.235607 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 132/280] 10.222.0.58 () {60 vars in 1132 bytes} [Tue May 17 19:10:09 2022] POST /identity/v3/users => generated 293 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:10:10.235607 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c06d371-93f3-4f26-a3c1-19c87c3dc75a None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:10.256450 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05df287-afbc-45bb-a640-574061900e2b None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:10.295296 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efa3da4-548a-4ea3-a7bc-79f47fef9adb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:10.296184 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efa3da4-548a-4ea3-a7bc-79f47fef9adb None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:10.296460 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efa3da4-548a-4ea3-a7bc-79f47fef9adb None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:10.296728 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efa3da4-548a-4ea3-a7bc-79f47fef9adb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:10.308924 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efa3da4-548a-4ea3-a7bc-79f47fef9adb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 17 19:10:10.316781 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-2efa3da4-548a-4ea3-a7bc-79f47fef9adb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 17 19:10:10.316781 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:10.316781 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:10.316781 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:10.316781 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:10.316781 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:10.316781 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:10.316781 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:10.316781 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:10.316781 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:10.316781 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:10.316781 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:10.316781 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 17 19:10:10.316781 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) May 17 19:10:10.316781 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 17 19:10:10.316781 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 17 19:10:10.317717 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 17 19:10:10.317717 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 17 19:10:10.317717 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 17 19:10:10.317717 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 17 19:10:10.317717 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:10.317717 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:10.317717 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:10.317717 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:10.317717 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:10.317717 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:10.317717 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:10.317717 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:10:10.317717 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:10.317717 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:10.317717 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:10.317717 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:10.318508 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:10.318508 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:10.318508 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:10.318508 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:10.318508 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 17 19:10:10.318508 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 17 19:10:10.318508 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 17 19:10:10.318508 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 17 19:10:10.318508 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 17 19:10:10.318508 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 17 19:10:10.318508 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 17 19:10:10.318508 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:10:10.318508 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 149/281] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 17 19:10:10 2022] GET /identity/v3/roles/reader => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:10:10.323961 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efa3da4-548a-4ea3-a7bc-79f47fef9adb None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:10.347081 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c06d371-93f3-4f26-a3c1-19c87c3dc75a None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:10.386068 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab3f028-0c1b-415f-a984-6cfebfb59f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:10.386740 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab3f028-0c1b-415f-a984-6cfebfb59f06 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:10.387019 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab3f028-0c1b-415f-a984-6cfebfb59f06 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:10.387290 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab3f028-0c1b-415f-a984-6cfebfb59f06 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:10.388173 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab3f028-0c1b-415f-a984-6cfebfb59f06 None admin] RBAC: Adding query filter params (role.id=114e6a16e63447d1924707ace5aace25, user.id=4a8932999b1f4466a222b252ca4ddb53, scope.system=all) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:10.388482 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab3f028-0c1b-415f-a984-6cfebfb59f06 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:10.389946 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab3f028-0c1b-415f-a984-6cfebfb59f06 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:10.395667 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 133/282] 10.222.0.58 () {56 vars in 1308 bytes} [Tue May 17 19:10:10 2022] GET /identity/v3/role_assignments?user.id=4a8932999b1f4466a222b252ca4ddb53&scope.system=all&role.id=114e6a16e63447d1924707ace5aace25 => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:10.436760 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efa3da4-548a-4ea3-a7bc-79f47fef9adb None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:10.476967 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bbe3b5-a397-421f-8070-f07ef2debb6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:10.477654 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bbe3b5-a397-421f-8070-f07ef2debb6c None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:10.477931 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bbe3b5-a397-421f-8070-f07ef2debb6c None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:10.478200 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bbe3b5-a397-421f-8070-f07ef2debb6c None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:10.478981 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bbe3b5-a397-421f-8070-f07ef2debb6c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:10.479285 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bbe3b5-a397-421f-8070-f07ef2debb6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:10.480760 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bbe3b5-a397-421f-8070-f07ef2debb6c None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:10.490640 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 150/283] 10.222.0.58 () {56 vars in 1101 bytes} [Tue May 17 19:10:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:10:10.496994 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab3f028-0c1b-415f-a984-6cfebfb59f06 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:10.608222 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab3f028-0c1b-415f-a984-6cfebfb59f06 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:10.643275 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0388e8-76c7-4ca3-b275-d7bafa5a3383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:10.644188 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0388e8-76c7-4ca3-b275-d7bafa5a3383 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:10.644469 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0388e8-76c7-4ca3-b275-d7bafa5a3383 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:10.644741 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0388e8-76c7-4ca3-b275-d7bafa5a3383 None admin] PATH_INFO: `/v3/users/9132c66a30d74f2c9d97e6443f313eea` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:10.645948 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0388e8-76c7-4ca3-b275-d7bafa5a3383 None admin] RBAC: Authorizing `identity:get_user(user_id=9132c66a30d74f2c9d97e6443f313eea)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:10.647959 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0388e8-76c7-4ca3-b275-d7bafa5a3383 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:10.649895 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 134/284] 10.222.0.58 () {56 vars in 1177 bytes} [Tue May 17 19:10:10 2022] GET /identity/v3/users/9132c66a30d74f2c9d97e6443f313eea => generated 297 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:10.655285 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bbe3b5-a397-421f-8070-f07ef2debb6c None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:10.765146 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bbe3b5-a397-421f-8070-f07ef2debb6c None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:10.799824 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a193249-316b-4db4-a6d4-f91673fa966d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:10.800688 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a193249-316b-4db4-a6d4-f91673fa966d None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:10.800966 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a193249-316b-4db4-a6d4-f91673fa966d None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:10.801239 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a193249-316b-4db4-a6d4-f91673fa966d None admin] PATH_INFO: `/v3/projects/625d29bdd4ad47ee92370dc3404fe414` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:10.802426 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a193249-316b-4db4-a6d4-f91673fa966d None admin] RBAC: Authorizing `identity:get_project(project_id=625d29bdd4ad47ee92370dc3404fe414)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:10.803955 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a193249-316b-4db4-a6d4-f91673fa966d None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:10.805975 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 151/285] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 17 19:10:10 2022] GET /identity/v3/projects/625d29bdd4ad47ee92370dc3404fe414 => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:10.811505 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0388e8-76c7-4ca3-b275-d7bafa5a3383 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:10.912385 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0388e8-76c7-4ca3-b275-d7bafa5a3383 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:10.956206 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e6d35c-df32-40bb-a55e-d9b4aba5b4ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:10.957130 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e6d35c-df32-40bb-a55e-d9b4aba5b4ba None admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:10.957408 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e6d35c-df32-40bb-a55e-d9b4aba5b4ba None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:10.957677 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e6d35c-df32-40bb-a55e-d9b4aba5b4ba None admin] PATH_INFO: `/v3/projects/625d29bdd4ad47ee92370dc3404fe414/users/9132c66a30d74f2c9d97e6443f313eea/roles/8e3500b266214ae2be189641a1b4dbf1` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:10.960273 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e6d35c-df32-40bb-a55e-d9b4aba5b4ba None admin] RBAC: Authorizing `identity:create_grant(project_id=625d29bdd4ad47ee92370dc3404fe414, user_id=9132c66a30d74f2c9d97e6443f313eea, role_id=8e3500b266214ae2be189641a1b4dbf1)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:10.961730 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e6d35c-df32-40bb-a55e-d9b4aba5b4ba None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:11.048164 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 135/286] 10.222.0.58 () {58 vars in 1439 bytes} [Tue May 17 19:10:10 2022] PUT /identity/v3/projects/625d29bdd4ad47ee92370dc3404fe414/users/9132c66a30d74f2c9d97e6443f313eea/roles/8e3500b266214ae2be189641a1b4dbf1 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:10:11.169717 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fabd34-fdfc-49f4-aa51-7beff8938c05 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:11.170042 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fabd34-fdfc-49f4-aa51-7beff8938c05 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:11.170338 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fabd34-fdfc-49f4-aa51-7beff8938c05 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:11.172050 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 152/287] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:10:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:10:11.178052 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a8ccad-565e-415b-ae2f-5963835abd30 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:11.178342 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a8ccad-565e-415b-ae2f-5963835abd30 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:11.178730 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a8ccad-565e-415b-ae2f-5963835abd30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:11.200653 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-96a8ccad-565e-415b-ae2f-5963835abd30 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:10:11.201886 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-96a8ccad-565e-415b-ae2f-5963835abd30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:10:11.292080 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 136/288] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:10:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 17 19:10:11.314837 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fabd34-fdfc-49f4-aa51-7beff8938c05 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:11.427385 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fabd34-fdfc-49f4-aa51-7beff8938c05 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:11.474015 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ac9259-7683-4d0b-a42b-c2238fddc71e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:11.475157 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ac9259-7683-4d0b-a42b-c2238fddc71e None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:11.475713 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ac9259-7683-4d0b-a42b-c2238fddc71e None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:11.476181 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ac9259-7683-4d0b-a42b-c2238fddc71e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:11.492230 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ac9259-7683-4d0b-a42b-c2238fddc71e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 17 19:10:11.500294 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-97ac9259-7683-4d0b-a42b-c2238fddc71e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 17 19:10:11.500294 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:11.500294 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:11.500294 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:11.500294 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:11.500294 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:11.500294 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:11.500294 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:11.500294 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:11.500294 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:11.500294 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:11.500294 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:11.500294 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 17 19:10:11.500294 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) May 17 19:10:11.500294 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 17 19:10:11.500294 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 17 19:10:11.501143 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 17 19:10:11.501143 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 17 19:10:11.501143 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 17 19:10:11.501143 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 17 19:10:11.501143 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:11.501143 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:11.501143 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:11.501143 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:11.501143 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:11.501143 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:11.501143 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:11.501143 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:10:11.501143 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:11.501143 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:11.501143 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:11.501143 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:11.501923 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:11.501923 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:11.501923 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:11.501923 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:11.501923 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 17 19:10:11.501923 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 17 19:10:11.501923 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 17 19:10:11.501923 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 17 19:10:11.501923 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 17 19:10:11.501923 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 17 19:10:11.501923 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 17 19:10:11.501923 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:10:11.504303 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 153/289] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 17 19:10:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:10:11.510815 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a8ccad-565e-415b-ae2f-5963835abd30 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:11.647153 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a8ccad-565e-415b-ae2f-5963835abd30 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:11.759431 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aced3a9-2973-4499-b920-9152c54284b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:11.760487 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aced3a9-2973-4499-b920-9152c54284b2 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:11.760965 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aced3a9-2973-4499-b920-9152c54284b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:11.761431 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aced3a9-2973-4499-b920-9152c54284b2 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:11.762358 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aced3a9-2973-4499-b920-9152c54284b2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:11.762878 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aced3a9-2973-4499-b920-9152c54284b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:11.764499 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aced3a9-2973-4499-b920-9152c54284b2 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:11.773335 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 137/290] 10.222.0.58 () {56 vars in 1101 bytes} [Tue May 17 19:10:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:11.781142 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ac9259-7683-4d0b-a42b-c2238fddc71e None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:11.888164 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ac9259-7683-4d0b-a42b-c2238fddc71e None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:11.922740 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cedc7c-aaca-4d1c-a342-056321ff2d97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:11.923956 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cedc7c-aaca-4d1c-a342-056321ff2d97 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:11.924435 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cedc7c-aaca-4d1c-a342-056321ff2d97 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:11.924899 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cedc7c-aaca-4d1c-a342-056321ff2d97 None admin] PATH_INFO: `/v3/users/614688dc21d94bc7a5f85a06df453136` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:11.926310 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cedc7c-aaca-4d1c-a342-056321ff2d97 None admin] RBAC: Authorizing `identity:get_user(user_id=614688dc21d94bc7a5f85a06df453136)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:11.928024 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cedc7c-aaca-4d1c-a342-056321ff2d97 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:11.930807 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 154/291] 10.222.0.58 () {56 vars in 1177 bytes} [Tue May 17 19:10:11 2022] GET /identity/v3/users/614688dc21d94bc7a5f85a06df453136 => generated 297 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:11.936421 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aced3a9-2973-4499-b920-9152c54284b2 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:12.037618 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aced3a9-2973-4499-b920-9152c54284b2 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:12.071767 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b576590f-ae79-49ce-9fed-08ffd2bd2109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:12.072414 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b576590f-ae79-49ce-9fed-08ffd2bd2109 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:12.072694 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b576590f-ae79-49ce-9fed-08ffd2bd2109 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:12.072965 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b576590f-ae79-49ce-9fed-08ffd2bd2109 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:12.139706 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b576590f-ae79-49ce-9fed-08ffd2bd2109 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:12.141242 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b576590f-ae79-49ce-9fed-08ffd2bd2109 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:12.240851 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 138/292] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 17 19:10:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 17 19:10:12.246378 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cedc7c-aaca-4d1c-a342-056321ff2d97 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:12.345413 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cedc7c-aaca-4d1c-a342-056321ff2d97 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:12.379449 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e399e892-4bba-43b3-b89b-2502c4443c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:12.380337 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e399e892-4bba-43b3-b89b-2502c4443c27 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:12.380611 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e399e892-4bba-43b3-b89b-2502c4443c27 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:12.380877 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e399e892-4bba-43b3-b89b-2502c4443c27 None admin] PATH_INFO: `/v3/projects/625d29bdd4ad47ee92370dc3404fe414` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:12.382090 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e399e892-4bba-43b3-b89b-2502c4443c27 None admin] RBAC: Authorizing `identity:get_project(project_id=625d29bdd4ad47ee92370dc3404fe414)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:12.383565 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e399e892-4bba-43b3-b89b-2502c4443c27 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:12.385331 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 155/293] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 17 19:10:12 2022] GET /identity/v3/projects/625d29bdd4ad47ee92370dc3404fe414 => generated 302 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:12.391017 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b576590f-ae79-49ce-9fed-08ffd2bd2109 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:12.489879 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b576590f-ae79-49ce-9fed-08ffd2bd2109 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:12.523985 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4b8eb9-e9f4-4207-83a3-3433578e2a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:12.524604 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4b8eb9-e9f4-4207-83a3-3433578e2a0e None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:12.524818 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4b8eb9-e9f4-4207-83a3-3433578e2a0e None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:12.525096 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4b8eb9-e9f4-4207-83a3-3433578e2a0e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:12.525924 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4b8eb9-e9f4-4207-83a3-3433578e2a0e None admin] RBAC: Adding query filter params (role.id=114e6a16e63447d1924707ace5aace25, scope.project.id=625d29bdd4ad47ee92370dc3404fe414, user.id=614688dc21d94bc7a5f85a06df453136) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:12.526231 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4b8eb9-e9f4-4207-83a3-3433578e2a0e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:12.527696 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4b8eb9-e9f4-4207-83a3-3433578e2a0e None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:12.533957 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 139/294] 10.222.0.58 () {56 vars in 1374 bytes} [Tue May 17 19:10:12 2022] GET /identity/v3/role_assignments?user.id=614688dc21d94bc7a5f85a06df453136&scope.project.id=625d29bdd4ad47ee92370dc3404fe414&role.id=114e6a16e63447d1924707ace5aace25 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:13.058363 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012ff440-69ae-4ccf-9277-7685d47cae81 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:13.058889 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012ff440-69ae-4ccf-9277-7685d47cae81 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:13.058889 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012ff440-69ae-4ccf-9277-7685d47cae81 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:13.060467 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 156/295] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:10:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:10:13.066680 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12980767-3946-48f4-8460-9192fcf348c8 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:13.066995 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12980767-3946-48f4-8460-9192fcf348c8 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:13.067236 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12980767-3946-48f4-8460-9192fcf348c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:13.087563 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-12980767-3946-48f4-8460-9192fcf348c8 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:10:13.088821 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-12980767-3946-48f4-8460-9192fcf348c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:10:13.162936 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 140/296] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:10:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 17 19:10:13.186504 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012ff440-69ae-4ccf-9277-7685d47cae81 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:13.254558 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec2e28c-3093-42d3-bea6-2b821af60d8e None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:13.254840 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec2e28c-3093-42d3-bea6-2b821af60d8e None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:13.255033 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec2e28c-3093-42d3-bea6-2b821af60d8e None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:13.258821 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 141/297] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:10:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 17 19:10:13.267333 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3858a63f-aa5d-42cb-ae1b-a9c1971b0d1a None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:13.267671 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3858a63f-aa5d-42cb-ae1b-a9c1971b0d1a None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:13.267954 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3858a63f-aa5d-42cb-ae1b-a9c1971b0d1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:13.288768 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-3858a63f-aa5d-42cb-ae1b-a9c1971b0d1a None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:10:13.290158 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-3858a63f-aa5d-42cb-ae1b-a9c1971b0d1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:10:13.301559 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012ff440-69ae-4ccf-9277-7685d47cae81 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:13.338158 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdab1b9-bc0e-4229-b710-e533291c632e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:13.338945 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdab1b9-bc0e-4229-b710-e533291c632e None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:13.339223 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdab1b9-bc0e-4229-b710-e533291c632e None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:13.339518 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdab1b9-bc0e-4229-b710-e533291c632e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:13.350787 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdab1b9-bc0e-4229-b710-e533291c632e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 17 19:10:13.358654 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-3fdab1b9-bc0e-4229-b710-e533291c632e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 17 19:10:13.358654 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:13.358654 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:13.358654 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:13.358654 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:13.358654 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:13.358654 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:13.358654 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:13.358654 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:13.358654 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:13.358654 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:13.358654 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:13.358654 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 17 19:10:13.358654 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) May 17 19:10:13.358654 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 17 19:10:13.358654 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 17 19:10:13.359558 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 17 19:10:13.359558 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 17 19:10:13.359558 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 17 19:10:13.359558 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 17 19:10:13.359558 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:13.359558 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:13.359558 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:13.359558 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:13.359558 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:13.359558 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:13.359558 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:13.359558 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:10:13.359558 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:13.359558 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:13.359558 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:13.359558 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:13.360331 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:13.360331 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:13.360331 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:13.360331 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:13.360331 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 17 19:10:13.360331 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 17 19:10:13.360331 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 17 19:10:13.360331 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 17 19:10:13.360331 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 17 19:10:13.360331 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 17 19:10:13.360331 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 17 19:10:13.360331 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:10:13.363600 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 142/298] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:10:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 17 19:10:13.365015 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fbb2fa-36f0-4d21-9287-f01186ddd019 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:13.365362 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fbb2fa-36f0-4d21-9287-f01186ddd019 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:13.365644 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fbb2fa-36f0-4d21-9287-f01186ddd019 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:13.367707 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 143/299] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:10:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:10:13.367874 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 157/300] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 17 19:10:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:10:13.373162 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fbb2fa-36f0-4d21-9287-f01186ddd019 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:13.373276 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77036b78-e373-4ff0-ae47-279f0c0ff411 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:13.373347 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77036b78-e373-4ff0-ae47-279f0c0ff411 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:13.375272 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77036b78-e373-4ff0-ae47-279f0c0ff411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:13.394232 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-77036b78-e373-4ff0-ae47-279f0c0ff411 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:10:13.395772 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-77036b78-e373-4ff0-ae47-279f0c0ff411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:10:13.465092 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 158/301] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:10:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 17 19:10:13.472645 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77036b78-e373-4ff0-ae47-279f0c0ff411 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:13.482672 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fbb2fa-36f0-4d21-9287-f01186ddd019 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:13.523525 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8460004b-f9cd-42bb-b36f-cd54ce41fc60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:13.524111 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8460004b-f9cd-42bb-b36f-cd54ce41fc60 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:13.524654 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8460004b-f9cd-42bb-b36f-cd54ce41fc60 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:13.524654 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8460004b-f9cd-42bb-b36f-cd54ce41fc60 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:13.525378 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8460004b-f9cd-42bb-b36f-cd54ce41fc60 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:13.525681 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8460004b-f9cd-42bb-b36f-cd54ce41fc60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:13.527081 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8460004b-f9cd-42bb-b36f-cd54ce41fc60 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:13.535139 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 144/302] 10.222.0.58 () {56 vars in 1101 bytes} [Tue May 17 19:10:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:13.536209 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8460004b-f9cd-42bb-b36f-cd54ce41fc60 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:13.574622 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77036b78-e373-4ff0-ae47-279f0c0ff411 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:13.610034 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21822227-25dd-42a6-8869-1251fd49fbe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:13.610821 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21822227-25dd-42a6-8869-1251fd49fbe8 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:13.611104 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21822227-25dd-42a6-8869-1251fd49fbe8 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:13.611373 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21822227-25dd-42a6-8869-1251fd49fbe8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:13.612578 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21822227-25dd-42a6-8869-1251fd49fbe8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:13.614012 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21822227-25dd-42a6-8869-1251fd49fbe8 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:13.619947 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 159/303] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:10:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:13.619947 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21822227-25dd-42a6-8869-1251fd49fbe8 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:13.645091 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8460004b-f9cd-42bb-b36f-cd54ce41fc60 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:13.680479 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b59a02d-3d7b-4601-9b0a-403728fccbc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:13.681268 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b59a02d-3d7b-4601-9b0a-403728fccbc0 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:13.681550 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b59a02d-3d7b-4601-9b0a-403728fccbc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:13.681824 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b59a02d-3d7b-4601-9b0a-403728fccbc0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:13.692775 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b59a02d-3d7b-4601-9b0a-403728fccbc0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 17 19:10:13.700413 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-6b59a02d-3d7b-4601-9b0a-403728fccbc0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 17 19:10:13.700413 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:13.700413 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:13.700413 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:13.700413 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:13.700413 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:13.700413 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:13.700413 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:13.700413 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:13.700413 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:13.700413 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:13.700413 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:13.700413 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 17 19:10:13.700413 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) May 17 19:10:13.700413 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 17 19:10:13.700413 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 17 19:10:13.701304 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 17 19:10:13.701304 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 17 19:10:13.701304 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 17 19:10:13.701304 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 17 19:10:13.701304 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:13.701304 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:13.701304 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:13.701304 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:13.701304 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:13.701304 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:13.701304 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:13.701304 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:10:13.701304 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:13.701304 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:13.701304 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:13.701304 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:13.702085 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:13.702085 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:13.702085 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:13.702085 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:13.702085 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 17 19:10:13.702085 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 17 19:10:13.702085 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 17 19:10:13.702085 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 17 19:10:13.702085 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 17 19:10:13.702085 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 17 19:10:13.702085 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 17 19:10:13.702085 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:10:13.702085 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 145/304] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 17 19:10:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:10:13.702735 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b59a02d-3d7b-4601-9b0a-403728fccbc0 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:13.719555 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21822227-25dd-42a6-8869-1251fd49fbe8 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:13.755592 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf9d7d7-c621-4ca1-a041-aca94ffe1e31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:13.756452 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf9d7d7-c621-4ca1-a041-aca94ffe1e31 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:13.756729 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf9d7d7-c621-4ca1-a041-aca94ffe1e31 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:13.757003 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf9d7d7-c621-4ca1-a041-aca94ffe1e31 None admin] PATH_INFO: `/v3/users/e0d0b874db204ff186c4f2fca2ae1a96` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:13.769359 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf9d7d7-c621-4ca1-a041-aca94ffe1e31 None admin] RBAC: Authorizing `identity:get_user(user_id=e0d0b874db204ff186c4f2fca2ae1a96)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:13.770819 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf9d7d7-c621-4ca1-a041-aca94ffe1e31 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:13.773423 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 160/305] 10.222.0.58 () {56 vars in 1177 bytes} [Tue May 17 19:10:13 2022] GET /identity/v3/users/e0d0b874db204ff186c4f2fca2ae1a96 => generated 293 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:13.774913 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf9d7d7-c621-4ca1-a041-aca94ffe1e31 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:13.806169 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b59a02d-3d7b-4601-9b0a-403728fccbc0 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:13.837249 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d563160-438a-444b-8eeb-053c8c093a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:13.837904 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d563160-438a-444b-8eeb-053c8c093a7a None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:13.838176 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d563160-438a-444b-8eeb-053c8c093a7a None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:13.838443 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d563160-438a-444b-8eeb-053c8c093a7a None admin] PATH_INFO: `/v3/groups` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:13.839143 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d563160-438a-444b-8eeb-053c8c093a7a None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:13.840595 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d563160-438a-444b-8eeb-053c8c093a7a None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:13.867999 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf9d7d7-c621-4ca1-a041-aca94ffe1e31 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:13.899785 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b492982f-ce70-4f68-b5b2-a4af2f7d319e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:13.900440 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b492982f-ce70-4f68-b5b2-a4af2f7d319e None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:13.900720 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b492982f-ce70-4f68-b5b2-a4af2f7d319e None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:13.900992 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b492982f-ce70-4f68-b5b2-a4af2f7d319e None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:13.901725 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b492982f-ce70-4f68-b5b2-a4af2f7d319e None admin] RBAC: Adding query filter params (name=member) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:13.902027 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b492982f-ce70-4f68-b5b2-a4af2f7d319e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:13.903379 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b492982f-ce70-4f68-b5b2-a4af2f7d319e None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:13.911181 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 161/306] 10.222.0.58 () {56 vars in 1101 bytes} [Tue May 17 19:10:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:13.912384 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4719e9d1-82e6-436e-b30a-638f6fc178c2 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:13.913242 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4719e9d1-82e6-436e-b30a-638f6fc178c2 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:13.913531 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4719e9d1-82e6-436e-b30a-638f6fc178c2 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:13.914776 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 162/307] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:10:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:10:13.915467 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4719e9d1-82e6-436e-b30a-638f6fc178c2 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:13.925502 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 146/308] 10.222.0.58 () {60 vars in 1134 bytes} [Tue May 17 19:10:13 2022] POST /identity/v3/groups => generated 229 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:10:13.925502 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d563160-438a-444b-8eeb-053c8c093a7a None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:14.026179 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4719e9d1-82e6-436e-b30a-638f6fc178c2 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:14.029106 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d563160-438a-444b-8eeb-053c8c093a7a None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:14.062029 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9eeebe-941e-4e0d-beb6-e94ecffed1f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:14.062720 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9eeebe-941e-4e0d-beb6-e94ecffed1f3 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:14.062960 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9eeebe-941e-4e0d-beb6-e94ecffed1f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:14.063274 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9eeebe-941e-4e0d-beb6-e94ecffed1f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:14.063927 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0eadfe-8522-41dd-8317-ba8c2263df0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:14.064135 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9eeebe-941e-4e0d-beb6-e94ecffed1f3 None admin] RBAC: Adding query filter params (role.id=8e3500b266214ae2be189641a1b4dbf1, user.id=e0d0b874db204ff186c4f2fca2ae1a96, scope.system=all) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:14.064398 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9eeebe-941e-4e0d-beb6-e94ecffed1f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:14.064809 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0eadfe-8522-41dd-8317-ba8c2263df0c None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:14.065084 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0eadfe-8522-41dd-8317-ba8c2263df0c None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:14.065358 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0eadfe-8522-41dd-8317-ba8c2263df0c None admin] PATH_INFO: `/v3/users/4a8932999b1f4466a222b252ca4ddb53` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:14.065960 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9eeebe-941e-4e0d-beb6-e94ecffed1f3 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:14.066616 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0eadfe-8522-41dd-8317-ba8c2263df0c None admin] RBAC: Authorizing `identity:get_user(user_id=4a8932999b1f4466a222b252ca4ddb53)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:14.068144 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0eadfe-8522-41dd-8317-ba8c2263df0c None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:14.070438 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 147/309] 10.222.0.58 () {56 vars in 1177 bytes} [Tue May 17 19:10:13 2022] GET /identity/v3/users/4a8932999b1f4466a222b252ca4ddb53 => generated 293 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:14.072496 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 163/310] 10.222.0.58 () {56 vars in 1308 bytes} [Tue May 17 19:10:13 2022] GET /identity/v3/role_assignments?user.id=e0d0b874db204ff186c4f2fca2ae1a96&scope.system=all&role.id=8e3500b266214ae2be189641a1b4dbf1 => generated 226 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:14.079536 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9eeebe-941e-4e0d-beb6-e94ecffed1f3 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:14.079751 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb6ecbd-c08b-4a85-895b-fe1b8921f708 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:14.079751 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb6ecbd-c08b-4a85-895b-fe1b8921f708 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:14.079751 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb6ecbd-c08b-4a85-895b-fe1b8921f708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:14.100702 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-6cb6ecbd-c08b-4a85-895b-fe1b8921f708 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:10:14.102147 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-6cb6ecbd-c08b-4a85-895b-fe1b8921f708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:10:14.193727 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 148/311] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:10:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 17 19:10:14.195517 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9eeebe-941e-4e0d-beb6-e94ecffed1f3 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:14.217917 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb6ecbd-c08b-4a85-895b-fe1b8921f708 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:14.237639 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f735a669-12f6-4a91-ab64-7701a4772882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:14.238937 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f735a669-12f6-4a91-ab64-7701a4772882 None admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:14.239514 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f735a669-12f6-4a91-ab64-7701a4772882 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:14.239988 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f735a669-12f6-4a91-ab64-7701a4772882 None admin] PATH_INFO: `/v3/system/users/4a8932999b1f4466a222b252ca4ddb53/roles/114e6a16e63447d1924707ace5aace25` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:14.241893 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f735a669-12f6-4a91-ab64-7701a4772882 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=4a8932999b1f4466a222b252ca4ddb53, role_id=114e6a16e63447d1924707ace5aace25)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:14.243523 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f735a669-12f6-4a91-ab64-7701a4772882 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:14.320460 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb6ecbd-c08b-4a85-895b-fe1b8921f708 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:14.348223 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 164/312] 10.222.0.58 () {58 vars in 1334 bytes} [Tue May 17 19:10:14 2022] PUT /identity/v3/system/users/4a8932999b1f4466a222b252ca4ddb53/roles/114e6a16e63447d1924707ace5aace25 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 17 19:10:14.356752 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af04f8fa-6949-4c1c-963b-c19b67e334c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:14.357935 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af04f8fa-6949-4c1c-963b-c19b67e334c3 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:14.358416 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af04f8fa-6949-4c1c-963b-c19b67e334c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:14.358900 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af04f8fa-6949-4c1c-963b-c19b67e334c3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:14.373376 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af04f8fa-6949-4c1c-963b-c19b67e334c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 17 19:10:14.382706 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-af04f8fa-6949-4c1c-963b-c19b67e334c3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 17 19:10:14.382706 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:14.382706 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:14.382706 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:14.382706 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:14.382706 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:14.382706 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:14.382706 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:14.382706 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:14.382706 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:14.382706 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:14.382706 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:14.382706 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 17 19:10:14.382706 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) May 17 19:10:14.382706 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 17 19:10:14.382706 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 17 19:10:14.390319 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 17 19:10:14.390319 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 17 19:10:14.390319 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 17 19:10:14.390319 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 17 19:10:14.390319 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:14.390319 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:14.390319 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:14.390319 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:14.390319 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:14.390319 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:14.390319 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:14.390319 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:10:14.390319 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:14.390319 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:14.390319 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:14.390319 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:14.391096 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:14.391096 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:14.391096 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:14.391096 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:14.391096 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 17 19:10:14.391096 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 17 19:10:14.391096 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 17 19:10:14.391096 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 17 19:10:14.391096 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 17 19:10:14.391096 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 17 19:10:14.391096 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 17 19:10:14.391096 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:10:14.391741 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f735a669-12f6-4a91-ab64-7701a4772882 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:14.392389 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 149/313] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 17 19:10:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 17 19:10:14.517652 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f735a669-12f6-4a91-ab64-7701a4772882 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:14.556807 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d975c8-3a66-4ab7-9df1-e70980bdd1ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:14.557828 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d975c8-3a66-4ab7-9df1-e70980bdd1ef None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:14.558317 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d975c8-3a66-4ab7-9df1-e70980bdd1ef None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:14.558832 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d975c8-3a66-4ab7-9df1-e70980bdd1ef None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:14.559865 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d975c8-3a66-4ab7-9df1-e70980bdd1ef None admin] RBAC: Adding query filter params (name=reader) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:14.560394 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d975c8-3a66-4ab7-9df1-e70980bdd1ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:14.561977 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d975c8-3a66-4ab7-9df1-e70980bdd1ef None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:14.571849 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 165/314] 10.222.0.58 () {56 vars in 1101 bytes} [Tue May 17 19:10:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:14.577278 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af04f8fa-6949-4c1c-963b-c19b67e334c3 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:14.680397 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af04f8fa-6949-4c1c-963b-c19b67e334c3 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:14.714659 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f749010-7eaf-4d75-9558-4047d3a10183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:14.715918 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f749010-7eaf-4d75-9558-4047d3a10183 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:14.716411 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f749010-7eaf-4d75-9558-4047d3a10183 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:14.716903 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f749010-7eaf-4d75-9558-4047d3a10183 None admin] PATH_INFO: `/v3/users/9132c66a30d74f2c9d97e6443f313eea` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:14.718321 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f749010-7eaf-4d75-9558-4047d3a10183 None admin] RBAC: Authorizing `identity:get_user(user_id=9132c66a30d74f2c9d97e6443f313eea)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:14.720019 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f749010-7eaf-4d75-9558-4047d3a10183 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:14.722406 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 150/315] 10.222.0.58 () {56 vars in 1177 bytes} [Tue May 17 19:10:14 2022] GET /identity/v3/users/9132c66a30d74f2c9d97e6443f313eea => generated 297 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:14.728077 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d975c8-3a66-4ab7-9df1-e70980bdd1ef None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:14.828263 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d975c8-3a66-4ab7-9df1-e70980bdd1ef None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:14.862219 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd4213d-c23d-4937-85fd-85e7abcf036a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:14.862880 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd4213d-c23d-4937-85fd-85e7abcf036a None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:14.863160 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd4213d-c23d-4937-85fd-85e7abcf036a None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:14.863429 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd4213d-c23d-4937-85fd-85e7abcf036a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:14.931602 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd4213d-c23d-4937-85fd-85e7abcf036a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:14.933062 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd4213d-c23d-4937-85fd-85e7abcf036a None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:15.036499 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 166/316] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 17 19:10:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 17 19:10:15.041911 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f749010-7eaf-4d75-9558-4047d3a10183 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:15.142621 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f749010-7eaf-4d75-9558-4047d3a10183 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:15.179005 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d98d594-38ad-4a64-8a68-5a87d867a0cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:15.180257 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d98d594-38ad-4a64-8a68-5a87d867a0cc None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:15.180725 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d98d594-38ad-4a64-8a68-5a87d867a0cc None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:15.181202 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d98d594-38ad-4a64-8a68-5a87d867a0cc None admin] PATH_INFO: `/v3/projects/625d29bdd4ad47ee92370dc3404fe414` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:15.182547 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d98d594-38ad-4a64-8a68-5a87d867a0cc None admin] RBAC: Authorizing `identity:get_project(project_id=625d29bdd4ad47ee92370dc3404fe414)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:15.184207 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d98d594-38ad-4a64-8a68-5a87d867a0cc None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:15.186369 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 151/317] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 17 19:10:15 2022] GET /identity/v3/projects/625d29bdd4ad47ee92370dc3404fe414 => generated 302 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:15.192350 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd4213d-c23d-4937-85fd-85e7abcf036a None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:15.297188 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd4213d-c23d-4937-85fd-85e7abcf036a None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:15.332127 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffe7452-3e5f-4dc2-96ec-4c6a59ae4315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:15.332777 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffe7452-3e5f-4dc2-96ec-4c6a59ae4315 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:15.333056 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffe7452-3e5f-4dc2-96ec-4c6a59ae4315 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:15.333324 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffe7452-3e5f-4dc2-96ec-4c6a59ae4315 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:15.334144 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffe7452-3e5f-4dc2-96ec-4c6a59ae4315 None admin] RBAC: Adding query filter params (role.id=8e3500b266214ae2be189641a1b4dbf1, scope.project.id=625d29bdd4ad47ee92370dc3404fe414, user.id=9132c66a30d74f2c9d97e6443f313eea) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:15.334474 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffe7452-3e5f-4dc2-96ec-4c6a59ae4315 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:15.335940 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffe7452-3e5f-4dc2-96ec-4c6a59ae4315 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:15.342142 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 167/318] 10.222.0.58 () {56 vars in 1374 bytes} [Tue May 17 19:10:15 2022] GET /identity/v3/role_assignments?user.id=9132c66a30d74f2c9d97e6443f313eea&scope.project.id=625d29bdd4ad47ee92370dc3404fe414&role.id=8e3500b266214ae2be189641a1b4dbf1 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:16.249198 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0e6d7c-a3cb-42c5-b3e6-98ce4b6aa94c None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:16.249684 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0e6d7c-a3cb-42c5-b3e6-98ce4b6aa94c None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:16.249684 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0e6d7c-a3cb-42c5-b3e6-98ce4b6aa94c None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:16.251145 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 152/319] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:10:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:10:16.256099 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfdf6fb-85fe-4eaf-a416-aa7814617eff None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:16.257085 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfdf6fb-85fe-4eaf-a416-aa7814617eff None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:16.257249 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfdf6fb-85fe-4eaf-a416-aa7814617eff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:16.273193 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-4cfdf6fb-85fe-4eaf-a416-aa7814617eff None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:10:16.274357 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-4cfdf6fb-85fe-4eaf-a416-aa7814617eff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:10:16.340158 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 168/320] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:10:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 17 19:10:16.345005 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bac971-fc8c-4cfc-8b30-7a9bfd79094a None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:16.345298 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bac971-fc8c-4cfc-8b30-7a9bfd79094a None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:16.345573 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bac971-fc8c-4cfc-8b30-7a9bfd79094a None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:16.347286 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 153/321] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:10:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:10:16.352815 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee40800a-0420-4356-962a-e604c01f4fbe None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:16.353105 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee40800a-0420-4356-962a-e604c01f4fbe None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:16.353381 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee40800a-0420-4356-962a-e604c01f4fbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:16.362301 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bac971-fc8c-4cfc-8b30-7a9bfd79094a None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:16.369942 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-ee40800a-0420-4356-962a-e604c01f4fbe None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:10:16.371119 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-ee40800a-0420-4356-962a-e604c01f4fbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:10:16.439130 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 169/322] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:10:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 17 19:10:16.462238 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee40800a-0420-4356-962a-e604c01f4fbe None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:16.470222 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bac971-fc8c-4cfc-8b30-7a9bfd79094a None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:16.504510 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fe8332-5305-44b8-b14d-0d442e4ac169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:16.505242 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fe8332-5305-44b8-b14d-0d442e4ac169 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:16.505526 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fe8332-5305-44b8-b14d-0d442e4ac169 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:16.505801 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fe8332-5305-44b8-b14d-0d442e4ac169 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:16.506958 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fe8332-5305-44b8-b14d-0d442e4ac169 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:16.508430 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fe8332-5305-44b8-b14d-0d442e4ac169 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:16.511530 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 154/323] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:10:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:10:16.515374 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fe8332-5305-44b8-b14d-0d442e4ac169 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:16.560273 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee40800a-0420-4356-962a-e604c01f4fbe None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:16.592656 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ce144c-c656-425b-93d3-79b6e83e0385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:16.593440 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ce144c-c656-425b-93d3-79b6e83e0385 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:16.593712 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ce144c-c656-425b-93d3-79b6e83e0385 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:16.593977 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ce144c-c656-425b-93d3-79b6e83e0385 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:16.604692 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ce144c-c656-425b-93d3-79b6e83e0385 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 17 19:10:16.612223 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-20ce144c-c656-425b-93d3-79b6e83e0385 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 17 19:10:16.612223 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:16.612223 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:16.612223 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:16.612223 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:16.612223 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:16.612223 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:16.612223 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:16.612223 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:16.612223 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:16.612223 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:16.612223 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:16.612223 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 17 19:10:16.612223 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) May 17 19:10:16.612223 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 17 19:10:16.612223 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 17 19:10:16.613127 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 17 19:10:16.613127 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 17 19:10:16.613127 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 17 19:10:16.613127 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 17 19:10:16.613127 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:16.613127 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:16.613127 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:16.613127 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:16.613127 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:16.613127 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:16.613127 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:16.613127 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:10:16.613127 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:16.613127 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:16.613127 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:16.613127 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:16.614413 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:16.614413 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:16.614413 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:16.614413 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:16.614413 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 17 19:10:16.614413 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 17 19:10:16.614413 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 17 19:10:16.614413 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 17 19:10:16.614413 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 17 19:10:16.614413 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 17 19:10:16.614413 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 17 19:10:16.614413 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:10:16.615353 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fe8332-5305-44b8-b14d-0d442e4ac169 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:16.615438 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 170/324] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 17 19:10:16 2022] GET /identity/v3/roles/reader => generated 84 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 17 19:10:16.615438 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646e0e2b-67d0-4aa1-8c43-16d5aacdbcb9 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:16.615438 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646e0e2b-67d0-4aa1-8c43-16d5aacdbcb9 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:16.615438 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646e0e2b-67d0-4aa1-8c43-16d5aacdbcb9 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:16.617065 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 171/325] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:10:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:10:16.619744 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646e0e2b-67d0-4aa1-8c43-16d5aacdbcb9 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:16.648592 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6821b8-d9bf-4dfb-b27c-f1660292725c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:16.649250 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6821b8-d9bf-4dfb-b27c-f1660292725c None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:16.649527 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6821b8-d9bf-4dfb-b27c-f1660292725c None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:16.649797 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6821b8-d9bf-4dfb-b27c-f1660292725c None admin] PATH_INFO: `/v3/groups` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:16.650472 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6821b8-d9bf-4dfb-b27c-f1660292725c None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:16.652010 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6821b8-d9bf-4dfb-b27c-f1660292725c None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:16.715068 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646e0e2b-67d0-4aa1-8c43-16d5aacdbcb9 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:16.746723 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d771506-0d06-42bb-9500-4538777b6fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:16.747367 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d771506-0d06-42bb-9500-4538777b6fba None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:16.747673 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d771506-0d06-42bb-9500-4538777b6fba None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:16.747944 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d771506-0d06-42bb-9500-4538777b6fba None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:16.748672 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d771506-0d06-42bb-9500-4538777b6fba None admin] RBAC: Adding query filter params (name=reader) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:16.749047 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d771506-0d06-42bb-9500-4538777b6fba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:16.750413 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d771506-0d06-42bb-9500-4538777b6fba None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:16.759224 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 172/326] 10.222.0.58 () {56 vars in 1101 bytes} [Tue May 17 19:10:16 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:10:16.760342 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90ea6ee-a7ec-4a44-861a-a0ee36891a94 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:16.760620 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90ea6ee-a7ec-4a44-861a-a0ee36891a94 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:16.760881 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90ea6ee-a7ec-4a44-861a-a0ee36891a94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:16.775987 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-d90ea6ee-a7ec-4a44-861a-a0ee36891a94 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:10:16.777137 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-d90ea6ee-a7ec-4a44-861a-a0ee36891a94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:10:16.803781 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 155/327] 10.222.0.58 () {60 vars in 1134 bytes} [Tue May 17 19:10:16 2022] POST /identity/v3/groups => generated 226 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 17 19:10:16.809089 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6821b8-d9bf-4dfb-b27c-f1660292725c None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:16.843393 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 173/328] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:10:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 17 19:10:16.870326 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90ea6ee-a7ec-4a44-861a-a0ee36891a94 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:16.912013 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6821b8-d9bf-4dfb-b27c-f1660292725c None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:16.947215 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019c0020-e574-4644-85b2-3f94ce7b9cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:16.948005 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019c0020-e574-4644-85b2-3f94ce7b9cd7 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:16.948282 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019c0020-e574-4644-85b2-3f94ce7b9cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:16.948543 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019c0020-e574-4644-85b2-3f94ce7b9cd7 None admin] PATH_INFO: `/v3/users/e0d0b874db204ff186c4f2fca2ae1a96` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:16.949792 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019c0020-e574-4644-85b2-3f94ce7b9cd7 None admin] RBAC: Authorizing `identity:get_user(user_id=e0d0b874db204ff186c4f2fca2ae1a96)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:16.951249 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019c0020-e574-4644-85b2-3f94ce7b9cd7 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:16.953253 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 156/329] 10.222.0.58 () {56 vars in 1177 bytes} [Tue May 17 19:10:16 2022] GET /identity/v3/users/e0d0b874db204ff186c4f2fca2ae1a96 => generated 293 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:16.958760 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019c0020-e574-4644-85b2-3f94ce7b9cd7 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:16.973436 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90ea6ee-a7ec-4a44-861a-a0ee36891a94 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:17.007287 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c1f8a2-31ae-4cbb-bdca-b051de579c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:17.007287 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c1f8a2-31ae-4cbb-bdca-b051de579c02 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:17.007776 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c1f8a2-31ae-4cbb-bdca-b051de579c02 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:17.007857 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c1f8a2-31ae-4cbb-bdca-b051de579c02 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:17.019684 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c1f8a2-31ae-4cbb-bdca-b051de579c02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 17 19:10:17.028868 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-e7c1f8a2-31ae-4cbb-bdca-b051de579c02 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 17 19:10:17.028868 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:17.028868 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:17.028868 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:17.028868 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:17.028868 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:17.028868 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:17.028868 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:17.028868 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:17.028868 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:17.028868 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:17.028868 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:17.028868 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 17 19:10:17.028868 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) May 17 19:10:17.028868 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 17 19:10:17.028868 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 17 19:10:17.029746 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 17 19:10:17.029746 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 17 19:10:17.029746 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 17 19:10:17.029746 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 17 19:10:17.029746 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:17.029746 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:17.029746 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:17.029746 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:17.029746 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:17.029746 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:17.029746 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:17.029746 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:10:17.029746 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:17.029746 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:17.029746 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:17.029746 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:17.031003 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:17.031003 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:17.031003 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:17.031003 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:17.031003 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 17 19:10:17.031003 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 17 19:10:17.031003 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 17 19:10:17.031003 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 17 19:10:17.031003 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 17 19:10:17.031003 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 17 19:10:17.031003 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 17 19:10:17.031003 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:10:17.031003 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 174/330] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 17 19:10:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 17 19:10:17.035718 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c1f8a2-31ae-4cbb-bdca-b051de579c02 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:17.062433 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019c0020-e574-4644-85b2-3f94ce7b9cd7 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:17.099179 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a098086-3d1f-4be7-9852-68fe81b2ffc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:17.100393 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a098086-3d1f-4be7-9852-68fe81b2ffc5 None admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:17.100843 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a098086-3d1f-4be7-9852-68fe81b2ffc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:17.101119 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a098086-3d1f-4be7-9852-68fe81b2ffc5 None admin] PATH_INFO: `/v3/system/users/e0d0b874db204ff186c4f2fca2ae1a96/roles/8e3500b266214ae2be189641a1b4dbf1` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:17.103522 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a098086-3d1f-4be7-9852-68fe81b2ffc5 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=e0d0b874db204ff186c4f2fca2ae1a96, role_id=8e3500b266214ae2be189641a1b4dbf1)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:17.104894 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a098086-3d1f-4be7-9852-68fe81b2ffc5 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:17.153252 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c1f8a2-31ae-4cbb-bdca-b051de579c02 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:17.170400 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 157/331] 10.222.0.58 () {58 vars in 1334 bytes} [Tue May 17 19:10:16 2022] PUT /identity/v3/system/users/e0d0b874db204ff186c4f2fca2ae1a96/roles/8e3500b266214ae2be189641a1b4dbf1 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 17 19:10:17.188424 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb31ebec-b252-4df4-81ca-636315bae967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:17.189119 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb31ebec-b252-4df4-81ca-636315bae967 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:17.189398 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb31ebec-b252-4df4-81ca-636315bae967 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:17.189672 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb31ebec-b252-4df4-81ca-636315bae967 None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:17.190445 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb31ebec-b252-4df4-81ca-636315bae967 None admin] RBAC: Adding query filter params (name=member) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:17.190780 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb31ebec-b252-4df4-81ca-636315bae967 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:17.192335 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb31ebec-b252-4df4-81ca-636315bae967 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:17.202674 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 175/332] 10.222.0.58 () {56 vars in 1101 bytes} [Tue May 17 19:10:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:10:17.208651 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a098086-3d1f-4be7-9852-68fe81b2ffc5 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:17.323336 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a098086-3d1f-4be7-9852-68fe81b2ffc5 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:17.360608 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ea4f36-2535-457d-bda9-20dcdedab77f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:17.361862 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ea4f36-2535-457d-bda9-20dcdedab77f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:17.362386 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ea4f36-2535-457d-bda9-20dcdedab77f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:17.362874 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ea4f36-2535-457d-bda9-20dcdedab77f None admin] PATH_INFO: `/v3/users/4a8932999b1f4466a222b252ca4ddb53` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:17.364462 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ea4f36-2535-457d-bda9-20dcdedab77f None admin] RBAC: Authorizing `identity:get_user(user_id=4a8932999b1f4466a222b252ca4ddb53)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:17.366284 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ea4f36-2535-457d-bda9-20dcdedab77f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:17.368579 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 158/333] 10.222.0.58 () {56 vars in 1177 bytes} [Tue May 17 19:10:17 2022] GET /identity/v3/users/4a8932999b1f4466a222b252ca4ddb53 => generated 293 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:17.374152 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb31ebec-b252-4df4-81ca-636315bae967 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:17.478595 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb31ebec-b252-4df4-81ca-636315bae967 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:17.513279 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff71d494-ec67-4bd5-bb68-75c45693db06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:17.514293 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff71d494-ec67-4bd5-bb68-75c45693db06 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:17.514777 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff71d494-ec67-4bd5-bb68-75c45693db06 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:17.515242 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff71d494-ec67-4bd5-bb68-75c45693db06 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:17.516323 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff71d494-ec67-4bd5-bb68-75c45693db06 None admin] RBAC: Adding query filter params (role.id=114e6a16e63447d1924707ace5aace25, user.id=4a8932999b1f4466a222b252ca4ddb53, scope.system=all) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:17.516840 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff71d494-ec67-4bd5-bb68-75c45693db06 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:17.518454 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff71d494-ec67-4bd5-bb68-75c45693db06 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:17.530697 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 176/334] 10.222.0.58 () {56 vars in 1308 bytes} [Tue May 17 19:10:17 2022] GET /identity/v3/role_assignments?user.id=4a8932999b1f4466a222b252ca4ddb53&scope.system=all&role.id=114e6a16e63447d1924707ace5aace25 => generated 510 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:10:19.238809 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f479577-0253-47cb-891e-1ba4d43f6a44 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:19.239830 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f479577-0253-47cb-891e-1ba4d43f6a44 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:19.240313 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f479577-0253-47cb-891e-1ba4d43f6a44 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:19.242650 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 159/335] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:10:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:10:19.248650 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e0b9f7-a05b-4e5f-a536-e7d3fc15ee0d None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:19.249550 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e0b9f7-a05b-4e5f-a536-e7d3fc15ee0d None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:19.250029 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e0b9f7-a05b-4e5f-a536-e7d3fc15ee0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:19.268566 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-44e0b9f7-a05b-4e5f-a536-e7d3fc15ee0d None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:10:19.270091 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-44e0b9f7-a05b-4e5f-a536-e7d3fc15ee0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:10:19.339165 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 177/336] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:10:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 17 19:10:19.366395 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f479577-0253-47cb-891e-1ba4d43f6a44 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:19.472175 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f479577-0253-47cb-891e-1ba4d43f6a44 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:19.507821 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e4d53f-9fb7-47f7-8b44-5f873ddd53d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:19.508966 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e4d53f-9fb7-47f7-8b44-5f873ddd53d1 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:19.509459 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e4d53f-9fb7-47f7-8b44-5f873ddd53d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:19.509926 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e4d53f-9fb7-47f7-8b44-5f873ddd53d1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:19.520731 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e4d53f-9fb7-47f7-8b44-5f873ddd53d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 17 19:10:19.529014 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-04e4d53f-9fb7-47f7-8b44-5f873ddd53d1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 17 19:10:19.529014 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:19.529014 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:19.529014 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:19.529014 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:19.529014 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:19.529014 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:19.529014 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:19.529014 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:19.529014 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:19.529014 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:19.529014 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:19.529014 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 17 19:10:19.529014 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) May 17 19:10:19.529014 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 17 19:10:19.529014 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 17 19:10:19.529982 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 17 19:10:19.529982 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 17 19:10:19.529982 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 17 19:10:19.529982 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 17 19:10:19.529982 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:19.529982 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:19.529982 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:19.529982 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:19.529982 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:19.529982 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:19.529982 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:19.529982 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:10:19.529982 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:19.529982 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:19.529982 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:19.529982 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:19.530770 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:19.530770 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:19.530770 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:19.530770 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:19.530770 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 17 19:10:19.530770 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 17 19:10:19.530770 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 17 19:10:19.530770 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 17 19:10:19.530770 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 17 19:10:19.530770 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 17 19:10:19.530770 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 17 19:10:19.530770 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:10:19.533704 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 160/337] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 17 19:10:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:10:19.540426 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e0b9f7-a05b-4e5f-a536-e7d3fc15ee0d None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:19.646180 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e0b9f7-a05b-4e5f-a536-e7d3fc15ee0d None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:19.681840 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6addac3e-f2c8-4785-98d2-27a1a9295764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:19.682851 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6addac3e-f2c8-4785-98d2-27a1a9295764 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:19.683369 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6addac3e-f2c8-4785-98d2-27a1a9295764 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:19.683884 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6addac3e-f2c8-4785-98d2-27a1a9295764 None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:19.684832 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6addac3e-f2c8-4785-98d2-27a1a9295764 None admin] RBAC: Adding query filter params (name=member) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:19.685382 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6addac3e-f2c8-4785-98d2-27a1a9295764 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:19.686977 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6addac3e-f2c8-4785-98d2-27a1a9295764 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:19.696068 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 178/338] 10.222.0.58 () {56 vars in 1101 bytes} [Tue May 17 19:10:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:19.701909 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e4d53f-9fb7-47f7-8b44-5f873ddd53d1 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:19.809627 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e4d53f-9fb7-47f7-8b44-5f873ddd53d1 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:19.847333 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad43742c-b8dc-4d25-bfeb-8b48a70b1f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:19.848393 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad43742c-b8dc-4d25-bfeb-8b48a70b1f9f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:19.848922 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad43742c-b8dc-4d25-bfeb-8b48a70b1f9f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:19.849397 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad43742c-b8dc-4d25-bfeb-8b48a70b1f9f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:19.937404 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a352423-2e2f-49ae-9e52-14f2c28ebc73 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:19.938058 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a352423-2e2f-49ae-9e52-14f2c28ebc73 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:19.938542 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a352423-2e2f-49ae-9e52-14f2c28ebc73 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:19.939017 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad43742c-b8dc-4d25-bfeb-8b48a70b1f9f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:19.940587 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad43742c-b8dc-4d25-bfeb-8b48a70b1f9f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:19.940796 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 179/339] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:10:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:10:19.957301 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1cf3c3-0be4-4e8c-b902-d9a80357fbdd None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:19.957866 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1cf3c3-0be4-4e8c-b902-d9a80357fbdd None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:19.958351 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1cf3c3-0be4-4e8c-b902-d9a80357fbdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:19.998881 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-9d1cf3c3-0be4-4e8c-b902-d9a80357fbdd None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:10:20.000454 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-9d1cf3c3-0be4-4e8c-b902-d9a80357fbdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:10:20.046891 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 161/340] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 17 19:10:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 17 19:10:20.052734 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad43742c-b8dc-4d25-bfeb-8b48a70b1f9f None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:20.089203 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 180/341] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:10:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 17 19:10:20.127394 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1cf3c3-0be4-4e8c-b902-d9a80357fbdd None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:20.229025 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad43742c-b8dc-4d25-bfeb-8b48a70b1f9f None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:20.271099 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1cf3c3-0be4-4e8c-b902-d9a80357fbdd None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:20.272722 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c004d6a9-9550-4740-a96a-33321f530244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:20.273585 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c004d6a9-9550-4740-a96a-33321f530244 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:20.273868 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c004d6a9-9550-4740-a96a-33321f530244 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:20.274134 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c004d6a9-9550-4740-a96a-33321f530244 None admin] PATH_INFO: `/v3/projects/d13e273646b44ecca3cd10cd867bbd04` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:20.279341 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c004d6a9-9550-4740-a96a-33321f530244 None admin] RBAC: Authorizing `identity:get_project(project_id=d13e273646b44ecca3cd10cd867bbd04)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:20.280850 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c004d6a9-9550-4740-a96a-33321f530244 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:20.282642 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 162/342] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 17 19:10:20 2022] GET /identity/v3/projects/d13e273646b44ecca3cd10cd867bbd04 => generated 298 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:10:20.283800 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e925c3-cc03-49f6-9fc7-48c68243fe8a None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:20.284079 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e925c3-cc03-49f6-9fc7-48c68243fe8a None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:20.284345 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e925c3-cc03-49f6-9fc7-48c68243fe8a None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:20.285799 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 163/343] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:10:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:10:20.292180 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e925c3-cc03-49f6-9fc7-48c68243fe8a None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:20.312340 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb84e939-0a0a-4c37-a144-ad1f1709f6b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:20.313376 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb84e939-0a0a-4c37-a144-ad1f1709f6b0 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:20.313797 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb84e939-0a0a-4c37-a144-ad1f1709f6b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:20.314237 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb84e939-0a0a-4c37-a144-ad1f1709f6b0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:20.325597 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb84e939-0a0a-4c37-a144-ad1f1709f6b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 17 19:10:20.333596 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-cb84e939-0a0a-4c37-a144-ad1f1709f6b0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 17 19:10:20.333596 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:20.333596 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:20.333596 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:20.333596 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:20.333596 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:20.333596 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:20.333596 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:20.333596 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:20.333596 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:20.333596 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:20.333596 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:20.333596 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 17 19:10:20.333596 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) May 17 19:10:20.333596 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 17 19:10:20.333596 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 17 19:10:20.335254 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 17 19:10:20.335254 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 17 19:10:20.335254 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 17 19:10:20.335254 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 17 19:10:20.335254 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:20.335254 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:20.335254 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:20.335254 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:20.335254 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:20.335254 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:20.335254 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:20.335254 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:10:20.335254 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:20.335254 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:20.335254 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:20.335254 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:20.336068 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:20.336068 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:20.336068 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:20.336068 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:20.336068 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 17 19:10:20.336068 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 17 19:10:20.336068 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 17 19:10:20.336068 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 17 19:10:20.336068 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 17 19:10:20.336068 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 17 19:10:20.336068 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 17 19:10:20.336068 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:10:20.338242 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 181/344] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 17 19:10:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:10:20.339731 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05e14da-7323-4312-935b-d1d7d2ce1542 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:20.340238 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05e14da-7323-4312-935b-d1d7d2ce1542 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:20.340682 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05e14da-7323-4312-935b-d1d7d2ce1542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:20.359976 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-c05e14da-7323-4312-935b-d1d7d2ce1542 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:10:20.361432 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-c05e14da-7323-4312-935b-d1d7d2ce1542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:10:20.433029 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 182/345] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:10:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 17 19:10:20.434140 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05e14da-7323-4312-935b-d1d7d2ce1542 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:20.456055 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e925c3-cc03-49f6-9fc7-48c68243fe8a None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:20.509679 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7952cb5b-7c71-42af-84a7-7d1e6e975114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:20.510592 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7952cb5b-7c71-42af-84a7-7d1e6e975114 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:20.510874 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7952cb5b-7c71-42af-84a7-7d1e6e975114 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:20.511212 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7952cb5b-7c71-42af-84a7-7d1e6e975114 None admin] PATH_INFO: `/v3/groups/7dae670cfaf346a2982f211f2796a7c7` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:20.519514 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7952cb5b-7c71-42af-84a7-7d1e6e975114 None admin] RBAC: Authorizing `identity:get_group(group_id=7dae670cfaf346a2982f211f2796a7c7)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:20.523324 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7952cb5b-7c71-42af-84a7-7d1e6e975114 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:20.525111 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 164/346] 10.222.0.58 () {56 vars in 1180 bytes} [Tue May 17 19:10:20 2022] GET /identity/v3/groups/7dae670cfaf346a2982f211f2796a7c7 => generated 226 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:10:20.525934 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7952cb5b-7c71-42af-84a7-7d1e6e975114 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:20.539275 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05e14da-7323-4312-935b-d1d7d2ce1542 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:20.576051 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125eceb7-0141-4ffb-b59e-70fc1f482e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:20.576965 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125eceb7-0141-4ffb-b59e-70fc1f482e91 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:20.577433 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125eceb7-0141-4ffb-b59e-70fc1f482e91 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:20.577868 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125eceb7-0141-4ffb-b59e-70fc1f482e91 None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:20.579545 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125eceb7-0141-4ffb-b59e-70fc1f482e91 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:20.580036 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125eceb7-0141-4ffb-b59e-70fc1f482e91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:20.581617 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125eceb7-0141-4ffb-b59e-70fc1f482e91 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:20.590660 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 183/347] 10.222.0.58 () {56 vars in 1101 bytes} [Tue May 17 19:10:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:20.591724 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125eceb7-0141-4ffb-b59e-70fc1f482e91 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:20.635202 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7952cb5b-7c71-42af-84a7-7d1e6e975114 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:20.669239 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ac90dc-0561-4128-acaf-667f724d4427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:20.670056 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ac90dc-0561-4128-acaf-667f724d4427 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:20.670331 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ac90dc-0561-4128-acaf-667f724d4427 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:20.670600 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ac90dc-0561-4128-acaf-667f724d4427 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:20.682589 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ac90dc-0561-4128-acaf-667f724d4427 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 17 19:10:20.690733 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-a3ac90dc-0561-4128-acaf-667f724d4427 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 17 19:10:20.690733 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:20.690733 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:20.690733 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:20.690733 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:20.690733 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:20.690733 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:20.690733 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:20.690733 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:20.690733 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:20.690733 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:20.690733 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:20.690733 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 17 19:10:20.690733 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) May 17 19:10:20.690733 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 17 19:10:20.690733 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 17 19:10:20.691917 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 17 19:10:20.691917 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 17 19:10:20.691917 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 17 19:10:20.691917 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 17 19:10:20.691917 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:20.691917 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:20.691917 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:20.691917 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:20.691917 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:20.691917 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:20.691917 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:20.691917 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:10:20.691917 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:20.691917 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:20.691917 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:20.691917 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:20.692929 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:20.692929 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:20.692929 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:20.692929 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:20.692929 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 17 19:10:20.692929 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 17 19:10:20.692929 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 17 19:10:20.692929 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 17 19:10:20.692929 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 17 19:10:20.692929 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 17 19:10:20.692929 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 17 19:10:20.692929 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:10:20.692929 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 165/348] 10.222.0.58 () {56 vars in 1114 bytes} [Tue May 17 19:10:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:10:20.694245 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ac90dc-0561-4128-acaf-667f724d4427 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:20.702878 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125eceb7-0141-4ffb-b59e-70fc1f482e91 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:20.738393 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca92f1a0-0cb8-47aa-b0d2-ec7e2c8e9770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:20.739330 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca92f1a0-0cb8-47aa-b0d2-ec7e2c8e9770 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:20.739784 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca92f1a0-0cb8-47aa-b0d2-ec7e2c8e9770 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:20.740200 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca92f1a0-0cb8-47aa-b0d2-ec7e2c8e9770 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:20.741710 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca92f1a0-0cb8-47aa-b0d2-ec7e2c8e9770 None admin] RBAC: Adding query filter params (group.id=7dae670cfaf346a2982f211f2796a7c7, role.id=114e6a16e63447d1924707ace5aace25, scope.project.id=d13e273646b44ecca3cd10cd867bbd04) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:20.742166 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca92f1a0-0cb8-47aa-b0d2-ec7e2c8e9770 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:20.743880 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca92f1a0-0cb8-47aa-b0d2-ec7e2c8e9770 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:20.749996 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 184/349] 10.222.0.58 () {56 vars in 1376 bytes} [Tue May 17 19:10:20 2022] GET /identity/v3/role_assignments?group.id=7dae670cfaf346a2982f211f2796a7c7&scope.project.id=d13e273646b44ecca3cd10cd867bbd04&role.id=114e6a16e63447d1924707ace5aace25 => generated 260 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:20.751049 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca92f1a0-0cb8-47aa-b0d2-ec7e2c8e9770 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:20.873915 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ac90dc-0561-4128-acaf-667f724d4427 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:20.877743 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca92f1a0-0cb8-47aa-b0d2-ec7e2c8e9770 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:20.914450 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f953002a-9250-446a-bcba-f47bcc98074f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:20.915406 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f953002a-9250-446a-bcba-f47bcc98074f None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:20.916025 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f953002a-9250-446a-bcba-f47bcc98074f None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:20.916451 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f953002a-9250-446a-bcba-f47bcc98074f None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:20.917379 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f953002a-9250-446a-bcba-f47bcc98074f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:20.917848 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f953002a-9250-446a-bcba-f47bcc98074f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:20.919596 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f953002a-9250-446a-bcba-f47bcc98074f None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:20.929449 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 185/350] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 17 19:10:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:20.935337 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f953002a-9250-446a-bcba-f47bcc98074f None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:20.948777 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3014297b-6830-46f1-ae2b-13812d9ec20f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:20.950557 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3014297b-6830-46f1-ae2b-13812d9ec20f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:20.953329 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3014297b-6830-46f1-ae2b-13812d9ec20f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:20.953753 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3014297b-6830-46f1-ae2b-13812d9ec20f None admin] PATH_INFO: `/v3/users/e0d0b874db204ff186c4f2fca2ae1a96` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:20.958034 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3014297b-6830-46f1-ae2b-13812d9ec20f None admin] RBAC: Authorizing `identity:get_user(user_id=e0d0b874db204ff186c4f2fca2ae1a96)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:20.958034 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3014297b-6830-46f1-ae2b-13812d9ec20f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:20.961293 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 166/351] 10.222.0.58 () {56 vars in 1177 bytes} [Tue May 17 19:10:20 2022] GET /identity/v3/users/e0d0b874db204ff186c4f2fca2ae1a96 => generated 293 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:20.966911 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3014297b-6830-46f1-ae2b-13812d9ec20f None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:21.066658 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f953002a-9250-446a-bcba-f47bcc98074f None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:21.108816 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03f5ca0-b528-4acf-99dd-3843a9bc08eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:21.109760 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03f5ca0-b528-4acf-99dd-3843a9bc08eb None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:21.110342 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03f5ca0-b528-4acf-99dd-3843a9bc08eb None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:21.113044 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03f5ca0-b528-4acf-99dd-3843a9bc08eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:21.191976 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03f5ca0-b528-4acf-99dd-3843a9bc08eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:21.193727 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03f5ca0-b528-4acf-99dd-3843a9bc08eb None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:21.194376 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3014297b-6830-46f1-ae2b-13812d9ec20f None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:21.261608 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a58b42-c67d-44ba-810d-e955926822da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:21.262312 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a58b42-c67d-44ba-810d-e955926822da None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:21.262594 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a58b42-c67d-44ba-810d-e955926822da None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:21.262867 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a58b42-c67d-44ba-810d-e955926822da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:21.263715 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a58b42-c67d-44ba-810d-e955926822da None admin] RBAC: Adding query filter params (role.id=8e3500b266214ae2be189641a1b4dbf1, user.id=e0d0b874db204ff186c4f2fca2ae1a96, scope.system=all) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:21.264024 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a58b42-c67d-44ba-810d-e955926822da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:21.265478 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a58b42-c67d-44ba-810d-e955926822da None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:21.287972 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 167/352] 10.222.0.58 () {56 vars in 1308 bytes} [Tue May 17 19:10:20 2022] GET /identity/v3/role_assignments?user.id=e0d0b874db204ff186c4f2fca2ae1a96&scope.system=all&role.id=8e3500b266214ae2be189641a1b4dbf1 => generated 510 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:21.289677 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e23f8c3-2baf-488f-9df0-0b086002d1ae None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:21.289796 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e23f8c3-2baf-488f-9df0-0b086002d1ae None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:21.290048 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e23f8c3-2baf-488f-9df0-0b086002d1ae None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:21.291593 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 168/353] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:10:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:10:21.297241 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1390fd5-d1d5-44be-b4f2-0f9e9d18f9b3 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:21.297429 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1390fd5-d1d5-44be-b4f2-0f9e9d18f9b3 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:21.297665 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1390fd5-d1d5-44be-b4f2-0f9e9d18f9b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:21.304504 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 186/354] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 17 19:10:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 17 19:10:21.310778 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03f5ca0-b528-4acf-99dd-3843a9bc08eb None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:21.319936 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-e1390fd5-d1d5-44be-b4f2-0f9e9d18f9b3 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:10:21.321174 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-e1390fd5-d1d5-44be-b4f2-0f9e9d18f9b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:10:21.408109 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 169/355] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:10:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 17 19:10:21.434191 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03f5ca0-b528-4acf-99dd-3843a9bc08eb None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:21.463374 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1390fd5-d1d5-44be-b4f2-0f9e9d18f9b3 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:21.476779 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeb802d-0dcf-4603-bb7b-d0c04908d664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:21.477944 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeb802d-0dcf-4603-bb7b-d0c04908d664 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:21.478365 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeb802d-0dcf-4603-bb7b-d0c04908d664 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:21.478775 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeb802d-0dcf-4603-bb7b-d0c04908d664 None admin] PATH_INFO: `/v3/projects/d13e273646b44ecca3cd10cd867bbd04` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:21.480914 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faeb802d-0dcf-4603-bb7b-d0c04908d664 None admin] RBAC: Authorizing `identity:get_project(project_id=d13e273646b44ecca3cd10cd867bbd04)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:21.483674 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faeb802d-0dcf-4603-bb7b-d0c04908d664 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:21.485983 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 187/356] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 17 19:10:21 2022] GET /identity/v3/projects/d13e273646b44ecca3cd10cd867bbd04 => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:21.496930 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeb802d-0dcf-4603-bb7b-d0c04908d664 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:21.615514 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1390fd5-d1d5-44be-b4f2-0f9e9d18f9b3 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:21.640797 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeb802d-0dcf-4603-bb7b-d0c04908d664 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:21.680574 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc193e9-613a-44da-830d-6395cac5fd8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:21.682073 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc193e9-613a-44da-830d-6395cac5fd8d None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:21.682498 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc193e9-613a-44da-830d-6395cac5fd8d None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:21.682909 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc193e9-613a-44da-830d-6395cac5fd8d None admin] PATH_INFO: `/v3/groups/7dae670cfaf346a2982f211f2796a7c7` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:21.684268 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc193e9-613a-44da-830d-6395cac5fd8d None admin] RBAC: Authorizing `identity:get_group(group_id=7dae670cfaf346a2982f211f2796a7c7)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:21.685855 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc193e9-613a-44da-830d-6395cac5fd8d None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:21.687764 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 188/357] 10.222.0.58 () {56 vars in 1180 bytes} [Tue May 17 19:10:21 2022] GET /identity/v3/groups/7dae670cfaf346a2982f211f2796a7c7 => generated 226 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:10:21.694299 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc193e9-613a-44da-830d-6395cac5fd8d None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:21.696045 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364d5347-3980-4538-9ed9-66d1a7bd84c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:21.699331 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364d5347-3980-4538-9ed9-66d1a7bd84c1 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:21.699777 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364d5347-3980-4538-9ed9-66d1a7bd84c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:21.700282 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364d5347-3980-4538-9ed9-66d1a7bd84c1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:21.726147 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364d5347-3980-4538-9ed9-66d1a7bd84c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 17 19:10:21.734110 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-364d5347-3980-4538-9ed9-66d1a7bd84c1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 17 19:10:21.734110 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:21.734110 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:21.734110 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:21.734110 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:21.734110 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:21.734110 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:21.734110 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:21.734110 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:21.734110 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:21.734110 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:21.734110 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:21.734110 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 17 19:10:21.734110 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) May 17 19:10:21.734110 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 17 19:10:21.734110 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 17 19:10:21.735080 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 17 19:10:21.735080 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 17 19:10:21.735080 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 17 19:10:21.735080 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 17 19:10:21.735080 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:21.735080 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:21.735080 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:21.735080 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:21.735080 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:21.735080 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:21.735080 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:21.735080 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:10:21.735080 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:21.735080 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:21.735080 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:21.735080 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:21.735978 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:21.735978 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:21.735978 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:21.735978 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:21.735978 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 17 19:10:21.735978 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 17 19:10:21.735978 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 17 19:10:21.735978 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 17 19:10:21.735978 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 17 19:10:21.735978 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 17 19:10:21.735978 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 17 19:10:21.735978 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:10:21.747290 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 170/358] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 17 19:10:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 17 19:10:21.753358 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364d5347-3980-4538-9ed9-66d1a7bd84c1 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:21.862154 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc193e9-613a-44da-830d-6395cac5fd8d None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:21.902607 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae8cc0f-83e2-4cae-b450-f77e699ad97e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:21.903447 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364d5347-3980-4538-9ed9-66d1a7bd84c1 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:21.903687 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae8cc0f-83e2-4cae-b450-f77e699ad97e None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:21.904126 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae8cc0f-83e2-4cae-b450-f77e699ad97e None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:21.904536 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae8cc0f-83e2-4cae-b450-f77e699ad97e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:21.905518 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae8cc0f-83e2-4cae-b450-f77e699ad97e None admin] RBAC: Adding query filter params (group.id=7dae670cfaf346a2982f211f2796a7c7, role.id=1055a3235b0143e58f1090c1e154e5c1, scope.project.id=d13e273646b44ecca3cd10cd867bbd04) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:21.905969 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae8cc0f-83e2-4cae-b450-f77e699ad97e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:21.909366 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae8cc0f-83e2-4cae-b450-f77e699ad97e None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:21.915462 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 189/359] 10.222.0.58 () {56 vars in 1376 bytes} [Tue May 17 19:10:21 2022] GET /identity/v3/role_assignments?group.id=7dae670cfaf346a2982f211f2796a7c7&scope.project.id=d13e273646b44ecca3cd10cd867bbd04&role.id=1055a3235b0143e58f1090c1e154e5c1 => generated 260 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:21.919993 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482eaef6-297d-46b5-bee6-08fc4ccd7b19 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:21.920581 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482eaef6-297d-46b5-bee6-08fc4ccd7b19 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:21.921060 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482eaef6-297d-46b5-bee6-08fc4ccd7b19 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:21.922689 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 190/360] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:10:21 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:10:21.929415 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae7c2e9-21cb-4394-b0df-d709d1707b33 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:21.930032 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae7c2e9-21cb-4394-b0df-d709d1707b33 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:21.930529 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae7c2e9-21cb-4394-b0df-d709d1707b33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:21.967023 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-aae7c2e9-21cb-4394-b0df-d709d1707b33 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:10:21.969316 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-aae7c2e9-21cb-4394-b0df-d709d1707b33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:10:21.978908 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd27a0ea-9dd0-407b-89f7-9439623481e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:21.980797 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd27a0ea-9dd0-407b-89f7-9439623481e7 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:21.981926 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd27a0ea-9dd0-407b-89f7-9439623481e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:21.982407 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd27a0ea-9dd0-407b-89f7-9439623481e7 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:21.987384 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd27a0ea-9dd0-407b-89f7-9439623481e7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:21.988024 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd27a0ea-9dd0-407b-89f7-9439623481e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:21.989671 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd27a0ea-9dd0-407b-89f7-9439623481e7 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:22.004760 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 171/361] 10.222.0.58 () {56 vars in 1101 bytes} [Tue May 17 19:10:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:22.011043 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd27a0ea-9dd0-407b-89f7-9439623481e7 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:22.080194 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 191/362] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:10:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 17 19:10:22.114904 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae7c2e9-21cb-4394-b0df-d709d1707b33 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:22.170107 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd27a0ea-9dd0-407b-89f7-9439623481e7 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:22.206896 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ae4162-a9e1-4f5b-b7cd-3371104358d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:22.207984 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ae4162-a9e1-4f5b-b7cd-3371104358d8 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:22.208493 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ae4162-a9e1-4f5b-b7cd-3371104358d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:22.208965 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ae4162-a9e1-4f5b-b7cd-3371104358d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:22.258495 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae7c2e9-21cb-4394-b0df-d709d1707b33 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:22.307115 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ae4162-a9e1-4f5b-b7cd-3371104358d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:22.309401 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ae4162-a9e1-4f5b-b7cd-3371104358d8 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:22.322804 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91787e98-62bf-4c83-a9c8-281677ca396c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:22.323617 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91787e98-62bf-4c83-a9c8-281677ca396c None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:22.323900 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91787e98-62bf-4c83-a9c8-281677ca396c None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:22.324174 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91787e98-62bf-4c83-a9c8-281677ca396c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:22.335054 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91787e98-62bf-4c83-a9c8-281677ca396c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 17 19:10:22.343293 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-91787e98-62bf-4c83-a9c8-281677ca396c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 17 19:10:22.343293 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:22.343293 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:22.343293 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:22.343293 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:22.343293 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:22.343293 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:22.343293 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:22.343293 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:22.343293 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:22.343293 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:22.343293 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:22.343293 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 17 19:10:22.343293 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) May 17 19:10:22.343293 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 17 19:10:22.343293 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 17 19:10:22.344212 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 17 19:10:22.344212 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 17 19:10:22.344212 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 17 19:10:22.344212 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 17 19:10:22.344212 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:22.344212 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:22.344212 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:22.344212 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:22.344212 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:22.344212 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:22.344212 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:22.344212 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:10:22.344212 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:22.344212 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:22.344212 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:22.344212 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:22.345004 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:22.345004 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:22.345004 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:22.345004 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:22.345004 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 17 19:10:22.345004 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 17 19:10:22.345004 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 17 19:10:22.345004 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 17 19:10:22.345004 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 17 19:10:22.345004 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 17 19:10:22.345004 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 17 19:10:22.345004 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:10:22.345004 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 192/363] 10.222.0.58 () {56 vars in 1114 bytes} [Tue May 17 19:10:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 17 19:10:22.347462 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca43ca68-0a60-46ac-9b74-1d4c32d21247 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:22.347462 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca43ca68-0a60-46ac-9b74-1d4c32d21247 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:22.347462 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca43ca68-0a60-46ac-9b74-1d4c32d21247 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:22.354045 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 193/364] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:10:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:10:22.354927 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca43ca68-0a60-46ac-9b74-1d4c32d21247 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:22.433102 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 172/365] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 17 19:10:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 17 19:10:22.434762 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59ae087-0b80-452b-936c-4c58cb25b633 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:22.435267 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59ae087-0b80-452b-936c-4c58cb25b633 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:22.435779 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59ae087-0b80-452b-936c-4c58cb25b633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:22.452192 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-e59ae087-0b80-452b-936c-4c58cb25b633 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:10:22.453774 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-e59ae087-0b80-452b-936c-4c58cb25b633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:10:22.463447 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca43ca68-0a60-46ac-9b74-1d4c32d21247 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:22.499383 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964cc295-9a83-4031-b848-b43a89ff541b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:22.500114 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964cc295-9a83-4031-b848-b43a89ff541b None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:22.500395 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964cc295-9a83-4031-b848-b43a89ff541b None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:22.500916 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964cc295-9a83-4031-b848-b43a89ff541b None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:22.501468 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964cc295-9a83-4031-b848-b43a89ff541b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:22.501773 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964cc295-9a83-4031-b848-b43a89ff541b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:22.503225 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964cc295-9a83-4031-b848-b43a89ff541b None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:22.513461 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 194/366] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 17 19:10:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:22.514228 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964cc295-9a83-4031-b848-b43a89ff541b None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:22.526004 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 173/367] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:10:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 17 19:10:22.527170 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59ae087-0b80-452b-936c-4c58cb25b633 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:22.641508 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59ae087-0b80-452b-936c-4c58cb25b633 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:22.649862 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964cc295-9a83-4031-b848-b43a89ff541b None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:22.676534 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29387e93-7a00-4d5c-9d26-e1f6da14fa20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:22.677193 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29387e93-7a00-4d5c-9d26-e1f6da14fa20 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:22.677475 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29387e93-7a00-4d5c-9d26-e1f6da14fa20 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:22.677780 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29387e93-7a00-4d5c-9d26-e1f6da14fa20 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:22.687067 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2397155-090e-4595-89b3-0ce4126394b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:22.688257 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2397155-090e-4595-89b3-0ce4126394b2 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:22.688761 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2397155-090e-4595-89b3-0ce4126394b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:22.689234 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2397155-090e-4595-89b3-0ce4126394b2 None admin] PATH_INFO: `/v3/projects/625d29bdd4ad47ee92370dc3404fe414` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:22.690624 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2397155-090e-4595-89b3-0ce4126394b2 None admin] RBAC: Authorizing `identity:get_project(project_id=625d29bdd4ad47ee92370dc3404fe414)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:22.692300 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2397155-090e-4595-89b3-0ce4126394b2 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:22.694619 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 195/368] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 17 19:10:22 2022] GET /identity/v3/projects/625d29bdd4ad47ee92370dc3404fe414 => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:22.695697 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2397155-090e-4595-89b3-0ce4126394b2 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:22.746458 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29387e93-7a00-4d5c-9d26-e1f6da14fa20 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:22.747943 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29387e93-7a00-4d5c-9d26-e1f6da14fa20 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:22.798072 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2397155-090e-4595-89b3-0ce4126394b2 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:22.833244 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66faa49e-8595-480f-b154-be72182a3a0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:22.834930 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66faa49e-8595-480f-b154-be72182a3a0a None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:22.835413 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66faa49e-8595-480f-b154-be72182a3a0a None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:22.835917 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66faa49e-8595-480f-b154-be72182a3a0a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:22.846861 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66faa49e-8595-480f-b154-be72182a3a0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 17 19:10:22.847553 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 174/369] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 17 19:10:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 17 19:10:22.848497 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29387e93-7a00-4d5c-9d26-e1f6da14fa20 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:22.859208 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-66faa49e-8595-480f-b154-be72182a3a0a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 17 19:10:22.859208 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:22.859208 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:22.859208 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:22.859208 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:22.859208 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:22.859208 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:22.859208 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:22.859208 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:22.859208 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:22.859208 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:22.859208 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:22.859208 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 17 19:10:22.859208 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) May 17 19:10:22.859208 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 17 19:10:22.859208 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 17 19:10:22.860209 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 17 19:10:22.860209 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 17 19:10:22.860209 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 17 19:10:22.860209 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 17 19:10:22.860209 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:22.860209 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:22.860209 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:22.860209 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:22.860209 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:22.860209 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:22.860209 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:22.860209 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:10:22.860209 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:22.860209 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:22.860209 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:22.860209 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:22.860987 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:22.860987 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:22.860987 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:22.860987 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:22.860987 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 17 19:10:22.860987 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 17 19:10:22.860987 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 17 19:10:22.860987 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 17 19:10:22.860987 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 17 19:10:22.860987 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 17 19:10:22.860987 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 17 19:10:22.860987 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:10:22.864771 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 196/370] 10.222.0.58 () {56 vars in 1096 bytes} [Tue May 17 19:10:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:10:22.865809 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66faa49e-8595-480f-b154-be72182a3a0a None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:22.978277 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29387e93-7a00-4d5c-9d26-e1f6da14fa20 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:22.999435 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66faa49e-8595-480f-b154-be72182a3a0a None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:23.019892 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f28879-2a14-4105-a599-016469ba8989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:23.021129 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f28879-2a14-4105-a599-016469ba8989 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:23.022184 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f28879-2a14-4105-a599-016469ba8989 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:23.022679 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f28879-2a14-4105-a599-016469ba8989 None admin] PATH_INFO: `/v3/groups/7dae670cfaf346a2982f211f2796a7c7` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:23.024111 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f28879-2a14-4105-a599-016469ba8989 None admin] RBAC: Authorizing `identity:get_group(group_id=7dae670cfaf346a2982f211f2796a7c7)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:23.025783 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f28879-2a14-4105-a599-016469ba8989 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:23.027990 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 175/371] 10.222.0.58 () {56 vars in 1180 bytes} [Tue May 17 19:10:22 2022] GET /identity/v3/groups/7dae670cfaf346a2982f211f2796a7c7 => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:23.028996 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f28879-2a14-4105-a599-016469ba8989 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:23.041297 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2148e47c-457e-4613-a326-3de320570dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:23.042075 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2148e47c-457e-4613-a326-3de320570dd6 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:23.042358 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2148e47c-457e-4613-a326-3de320570dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:23.042633 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2148e47c-457e-4613-a326-3de320570dd6 None admin] PATH_INFO: `/v3/projects/625d29bdd4ad47ee92370dc3404fe414` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:23.043852 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2148e47c-457e-4613-a326-3de320570dd6 None admin] RBAC: Authorizing `identity:get_project(project_id=625d29bdd4ad47ee92370dc3404fe414)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:23.045296 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2148e47c-457e-4613-a326-3de320570dd6 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:23.046999 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 197/372] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 17 19:10:22 2022] GET /identity/v3/projects/625d29bdd4ad47ee92370dc3404fe414 => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:23.047961 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2148e47c-457e-4613-a326-3de320570dd6 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:23.133926 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f28879-2a14-4105-a599-016469ba8989 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:23.151769 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2148e47c-457e-4613-a326-3de320570dd6 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:23.169356 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983444af-0c90-4f2f-a7c0-0bd3258a5223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:23.170442 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983444af-0c90-4f2f-a7c0-0bd3258a5223 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:23.170942 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983444af-0c90-4f2f-a7c0-0bd3258a5223 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:23.171479 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983444af-0c90-4f2f-a7c0-0bd3258a5223 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:23.172468 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983444af-0c90-4f2f-a7c0-0bd3258a5223 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:23.172970 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983444af-0c90-4f2f-a7c0-0bd3258a5223 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:23.174652 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983444af-0c90-4f2f-a7c0-0bd3258a5223 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:23.184299 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 176/373] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 17 19:10:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:23.185420 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983444af-0c90-4f2f-a7c0-0bd3258a5223 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:23.189777 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5788b3eb-4624-4e78-92d1-d63f75b49fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:23.189934 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5788b3eb-4624-4e78-92d1-d63f75b49fce None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:23.190160 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5788b3eb-4624-4e78-92d1-d63f75b49fce None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:23.190380 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5788b3eb-4624-4e78-92d1-d63f75b49fce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:23.191218 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5788b3eb-4624-4e78-92d1-d63f75b49fce None admin] RBAC: Adding query filter params (group.id=7dae670cfaf346a2982f211f2796a7c7, role.id=114e6a16e63447d1924707ace5aace25, scope.project.id=625d29bdd4ad47ee92370dc3404fe414) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:23.191554 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5788b3eb-4624-4e78-92d1-d63f75b49fce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:23.193031 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5788b3eb-4624-4e78-92d1-d63f75b49fce None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:23.199595 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 198/374] 10.222.0.58 () {56 vars in 1376 bytes} [Tue May 17 19:10:23 2022] GET /identity/v3/role_assignments?group.id=7dae670cfaf346a2982f211f2796a7c7&scope.project.id=625d29bdd4ad47ee92370dc3404fe414&role.id=114e6a16e63447d1924707ace5aace25 => generated 260 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:23.202067 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5788b3eb-4624-4e78-92d1-d63f75b49fce None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:23.311920 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983444af-0c90-4f2f-a7c0-0bd3258a5223 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:23.361449 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5788b3eb-4624-4e78-92d1-d63f75b49fce None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:23.371478 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eba4b5-1ec2-47c3-9760-39c987e8c6f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:23.372371 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eba4b5-1ec2-47c3-9760-39c987e8c6f8 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:23.372646 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eba4b5-1ec2-47c3-9760-39c987e8c6f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:23.372915 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eba4b5-1ec2-47c3-9760-39c987e8c6f8 None admin] PATH_INFO: `/v3/groups/7dae670cfaf346a2982f211f2796a7c7` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:23.374635 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65eba4b5-1ec2-47c3-9760-39c987e8c6f8 None admin] RBAC: Authorizing `identity:get_group(group_id=7dae670cfaf346a2982f211f2796a7c7)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:23.376232 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65eba4b5-1ec2-47c3-9760-39c987e8c6f8 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:23.377937 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 177/375] 10.222.0.58 () {56 vars in 1180 bytes} [Tue May 17 19:10:23 2022] GET /identity/v3/groups/7dae670cfaf346a2982f211f2796a7c7 => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:23.384646 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eba4b5-1ec2-47c3-9760-39c987e8c6f8 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:23.433577 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3a978d-0948-46c5-b9c4-b7e5d55699ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:23.434252 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3a978d-0948-46c5-b9c4-b7e5d55699ae None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:23.434530 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3a978d-0948-46c5-b9c4-b7e5d55699ae None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:23.434803 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3a978d-0948-46c5-b9c4-b7e5d55699ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:23.512078 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3a978d-0948-46c5-b9c4-b7e5d55699ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:23.513592 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3a978d-0948-46c5-b9c4-b7e5d55699ae None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:23.521068 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eba4b5-1ec2-47c3-9760-39c987e8c6f8 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:23.555117 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120f42ef-5b7a-4b14-bf4d-1b4acc5e3989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:23.555803 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120f42ef-5b7a-4b14-bf4d-1b4acc5e3989 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:23.556080 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120f42ef-5b7a-4b14-bf4d-1b4acc5e3989 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:23.556352 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120f42ef-5b7a-4b14-bf4d-1b4acc5e3989 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:23.557176 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120f42ef-5b7a-4b14-bf4d-1b4acc5e3989 None admin] RBAC: Adding query filter params (group.id=7dae670cfaf346a2982f211f2796a7c7, role.id=1055a3235b0143e58f1090c1e154e5c1, scope.project.id=625d29bdd4ad47ee92370dc3404fe414) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:23.557486 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120f42ef-5b7a-4b14-bf4d-1b4acc5e3989 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:23.558924 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120f42ef-5b7a-4b14-bf4d-1b4acc5e3989 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:23.564796 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 178/376] 10.222.0.58 () {56 vars in 1376 bytes} [Tue May 17 19:10:23 2022] GET /identity/v3/role_assignments?group.id=7dae670cfaf346a2982f211f2796a7c7&scope.project.id=625d29bdd4ad47ee92370dc3404fe414&role.id=1055a3235b0143e58f1090c1e154e5c1 => generated 260 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:10:23.628724 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 199/377] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 17 19:10:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 17 19:10:23.635285 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120f42ef-5b7a-4b14-bf4d-1b4acc5e3989 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:23.741842 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120f42ef-5b7a-4b14-bf4d-1b4acc5e3989 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:23.781688 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836a6379-d346-4bca-9a9d-49e0eadf9f05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:23.782589 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836a6379-d346-4bca-9a9d-49e0eadf9f05 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:23.782936 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836a6379-d346-4bca-9a9d-49e0eadf9f05 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:23.783214 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836a6379-d346-4bca-9a9d-49e0eadf9f05 None admin] PATH_INFO: `/v3/projects/07bd98f7df3a477a8bf75754083485f0` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:23.788408 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836a6379-d346-4bca-9a9d-49e0eadf9f05 None admin] RBAC: Authorizing `identity:get_project(project_id=07bd98f7df3a477a8bf75754083485f0)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:23.789939 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836a6379-d346-4bca-9a9d-49e0eadf9f05 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:23.792075 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 179/378] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 17 19:10:23 2022] GET /identity/v3/projects/07bd98f7df3a477a8bf75754083485f0 => generated 344 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:23.797671 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3a978d-0948-46c5-b9c4-b7e5d55699ae None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:23.811008 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004b7aa9-0466-46b7-9b86-b48ac85329fa None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:23.811326 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004b7aa9-0466-46b7-9b86-b48ac85329fa None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:23.811579 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004b7aa9-0466-46b7-9b86-b48ac85329fa None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:23.816067 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 180/379] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:10:23 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:10:23.819285 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a8ef87-2f7d-4118-a462-091d3f542d1f None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:23.819599 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a8ef87-2f7d-4118-a462-091d3f542d1f None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:23.819875 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a8ef87-2f7d-4118-a462-091d3f542d1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:23.848924 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-03a8ef87-2f7d-4118-a462-091d3f542d1f None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:10:23.850869 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-03a8ef87-2f7d-4118-a462-091d3f542d1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:10:23.942091 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 181/380] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:10:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 17 19:10:23.966326 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a8ef87-2f7d-4118-a462-091d3f542d1f None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:24.019029 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3a978d-0948-46c5-b9c4-b7e5d55699ae None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:24.061584 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0e0e6a-9b4e-4c8d-9733-b2e63f471ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:24.062449 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0e0e6a-9b4e-4c8d-9733-b2e63f471ad3 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:24.062726 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0e0e6a-9b4e-4c8d-9733-b2e63f471ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:24.062997 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0e0e6a-9b4e-4c8d-9733-b2e63f471ad3 None admin] PATH_INFO: `/v3/groups/87c9fe75443848f894d3e8f7b86160ca` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:24.069643 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0e0e6a-9b4e-4c8d-9733-b2e63f471ad3 None admin] RBAC: Authorizing `identity:get_group(group_id=87c9fe75443848f894d3e8f7b86160ca)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:24.071149 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0e0e6a-9b4e-4c8d-9733-b2e63f471ad3 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:24.073177 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 200/381] 10.222.0.58 () {56 vars in 1180 bytes} [Tue May 17 19:10:23 2022] GET /identity/v3/groups/87c9fe75443848f894d3e8f7b86160ca => generated 229 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:24.079000 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0e0e6a-9b4e-4c8d-9733-b2e63f471ad3 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:24.147790 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a8ef87-2f7d-4118-a462-091d3f542d1f None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:24.195566 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0e0e6a-9b4e-4c8d-9733-b2e63f471ad3 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:24.226690 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52fdd1a-1fc6-43e5-ba15-43748bdb00b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:24.227525 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52fdd1a-1fc6-43e5-ba15-43748bdb00b4 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:24.227778 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52fdd1a-1fc6-43e5-ba15-43748bdb00b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:24.228047 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52fdd1a-1fc6-43e5-ba15-43748bdb00b4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:24.245474 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52fdd1a-1fc6-43e5-ba15-43748bdb00b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 17 19:10:24.251446 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639eb9cf-5ce9-44a8-8075-37f37c4ea3e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:24.252117 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639eb9cf-5ce9-44a8-8075-37f37c4ea3e8 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:24.252392 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639eb9cf-5ce9-44a8-8075-37f37c4ea3e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:24.252663 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639eb9cf-5ce9-44a8-8075-37f37c4ea3e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:24.253489 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639eb9cf-5ce9-44a8-8075-37f37c4ea3e8 None admin] RBAC: Adding query filter params (group.id=87c9fe75443848f894d3e8f7b86160ca, role.id=60af5088793e4379b88821a86552a344, scope.project.id=07bd98f7df3a477a8bf75754083485f0) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:24.253791 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639eb9cf-5ce9-44a8-8075-37f37c4ea3e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:24.255238 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639eb9cf-5ce9-44a8-8075-37f37c4ea3e8 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:24.261880 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-f52fdd1a-1fc6-43e5-ba15-43748bdb00b4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 17 19:10:24.261880 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:24.261880 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:24.261880 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:24.261880 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:24.261880 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:24.261880 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:24.261880 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:24.261880 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:24.261880 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:24.261880 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:24.261880 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:24.261880 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 17 19:10:24.261880 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) May 17 19:10:24.261880 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 17 19:10:24.261880 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 17 19:10:24.262747 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 17 19:10:24.262747 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 17 19:10:24.262747 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 17 19:10:24.262747 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 17 19:10:24.262747 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:24.262747 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:24.262747 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:24.262747 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:24.262747 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:24.262747 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:24.262747 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:24.262747 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:10:24.262747 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:24.262747 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:24.262747 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:24.262747 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:24.264806 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:24.264806 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:24.264806 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:24.264806 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:24.264806 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 17 19:10:24.264806 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 17 19:10:24.264806 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 17 19:10:24.264806 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 17 19:10:24.264806 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 17 19:10:24.264806 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 17 19:10:24.264806 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 17 19:10:24.264806 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:10:24.264806 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 182/382] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 17 19:10:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:10:24.269676 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52fdd1a-1fc6-43e5-ba15-43748bdb00b4 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:24.274773 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 201/383] 10.222.0.58 () {56 vars in 1376 bytes} [Tue May 17 19:10:24 2022] GET /identity/v3/role_assignments?group.id=87c9fe75443848f894d3e8f7b86160ca&scope.project.id=07bd98f7df3a477a8bf75754083485f0&role.id=60af5088793e4379b88821a86552a344 => generated 260 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:24.417363 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52fdd1a-1fc6-43e5-ba15-43748bdb00b4 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:24.468131 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e85771-7612-46f7-9709-bf592ef4d638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:24.468725 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e85771-7612-46f7-9709-bf592ef4d638 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:24.468999 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e85771-7612-46f7-9709-bf592ef4d638 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:24.469266 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e85771-7612-46f7-9709-bf592ef4d638 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:24.470014 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e85771-7612-46f7-9709-bf592ef4d638 None admin] RBAC: Adding query filter params (name=member) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:24.470311 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e85771-7612-46f7-9709-bf592ef4d638 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:24.471992 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e85771-7612-46f7-9709-bf592ef4d638 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:24.488000 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 183/384] 10.222.0.58 () {56 vars in 1101 bytes} [Tue May 17 19:10:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:10:24.494493 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639eb9cf-5ce9-44a8-8075-37f37c4ea3e8 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:24.646053 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639eb9cf-5ce9-44a8-8075-37f37c4ea3e8 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:24.696250 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c20e926-a006-42d3-b586-19b67ad3358c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:24.697499 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c20e926-a006-42d3-b586-19b67ad3358c None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:24.698010 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c20e926-a006-42d3-b586-19b67ad3358c None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:24.698506 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c20e926-a006-42d3-b586-19b67ad3358c None admin] PATH_INFO: `/v3/groups/7dae670cfaf346a2982f211f2796a7c7` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:24.700015 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c20e926-a006-42d3-b586-19b67ad3358c None admin] RBAC: Authorizing `identity:get_group(group_id=7dae670cfaf346a2982f211f2796a7c7)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:24.701687 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c20e926-a006-42d3-b586-19b67ad3358c None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:24.703558 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 202/385] 10.222.0.58 () {56 vars in 1180 bytes} [Tue May 17 19:10:24 2022] GET /identity/v3/groups/7dae670cfaf346a2982f211f2796a7c7 => generated 226 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:24.709306 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e85771-7612-46f7-9709-bf592ef4d638 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:24.808670 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9134dd-e77d-4341-88e9-23762fce11ac None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:24.809279 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9134dd-e77d-4341-88e9-23762fce11ac None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:24.809774 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9134dd-e77d-4341-88e9-23762fce11ac None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:24.811338 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 203/386] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:10:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:10:24.829030 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee16257c-01cd-40de-9cc9-ead535dbfdcd None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:24.829322 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee16257c-01cd-40de-9cc9-ead535dbfdcd None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:24.829579 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee16257c-01cd-40de-9cc9-ead535dbfdcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:24.835568 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e85771-7612-46f7-9709-bf592ef4d638 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:24.863519 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-ee16257c-01cd-40de-9cc9-ead535dbfdcd None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:10:24.863519 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-ee16257c-01cd-40de-9cc9-ead535dbfdcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:10:24.870716 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49605dbf-0fa0-403f-af94-8fd0acad75f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:24.871692 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49605dbf-0fa0-403f-af94-8fd0acad75f5 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:24.871971 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49605dbf-0fa0-403f-af94-8fd0acad75f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:24.872241 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49605dbf-0fa0-403f-af94-8fd0acad75f5 None admin] PATH_INFO: `/v3/projects/d13e273646b44ecca3cd10cd867bbd04` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:24.873502 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49605dbf-0fa0-403f-af94-8fd0acad75f5 None admin] RBAC: Authorizing `identity:get_project(project_id=d13e273646b44ecca3cd10cd867bbd04)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:24.874974 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49605dbf-0fa0-403f-af94-8fd0acad75f5 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:24.877123 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 184/387] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 17 19:10:24 2022] GET /identity/v3/projects/d13e273646b44ecca3cd10cd867bbd04 => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:24.884136 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49605dbf-0fa0-403f-af94-8fd0acad75f5 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:25.002734 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49605dbf-0fa0-403f-af94-8fd0acad75f5 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:25.037911 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 204/388] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:10:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 17 19:10:25.041986 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523bb477-c5a4-4619-bd65-4183a9be797b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:25.043253 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523bb477-c5a4-4619-bd65-4183a9be797b None admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:25.043779 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523bb477-c5a4-4619-bd65-4183a9be797b None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:25.044288 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523bb477-c5a4-4619-bd65-4183a9be797b None admin] PATH_INFO: `/v3/projects/d13e273646b44ecca3cd10cd867bbd04/groups/7dae670cfaf346a2982f211f2796a7c7/roles/114e6a16e63447d1924707ace5aace25` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:25.046685 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523bb477-c5a4-4619-bd65-4183a9be797b None admin] RBAC: Authorizing `identity:create_grant(project_id=d13e273646b44ecca3cd10cd867bbd04, group_id=7dae670cfaf346a2982f211f2796a7c7, role_id=114e6a16e63447d1924707ace5aace25)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:25.048477 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523bb477-c5a4-4619-bd65-4183a9be797b None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:25.076877 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee16257c-01cd-40de-9cc9-ead535dbfdcd None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:25.131705 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 185/389] 10.222.0.58 () {58 vars in 1442 bytes} [Tue May 17 19:10:24 2022] PUT /identity/v3/projects/d13e273646b44ecca3cd10cd867bbd04/groups/7dae670cfaf346a2982f211f2796a7c7/roles/114e6a16e63447d1924707ace5aace25 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 17 19:10:25.194352 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee16257c-01cd-40de-9cc9-ead535dbfdcd None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:25.233370 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcf4fd7-c0f9-4303-b54f-e68ef5d3b20a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:25.234176 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcf4fd7-c0f9-4303-b54f-e68ef5d3b20a None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:25.234449 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcf4fd7-c0f9-4303-b54f-e68ef5d3b20a None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:25.234771 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcf4fd7-c0f9-4303-b54f-e68ef5d3b20a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:25.245647 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcf4fd7-c0f9-4303-b54f-e68ef5d3b20a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 17 19:10:25.254022 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-5dcf4fd7-c0f9-4303-b54f-e68ef5d3b20a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 17 19:10:25.254022 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:25.254022 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:25.254022 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:25.254022 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:25.254022 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:25.254022 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:25.254022 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:25.254022 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:25.254022 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:25.254022 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:25.254022 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:25.254022 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 17 19:10:25.254022 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) May 17 19:10:25.254022 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 17 19:10:25.254022 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 17 19:10:25.255008 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 17 19:10:25.255008 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 17 19:10:25.255008 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 17 19:10:25.255008 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 17 19:10:25.255008 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:25.255008 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:25.255008 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:25.255008 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:25.255008 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:25.255008 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:25.255008 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:25.255008 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:10:25.255008 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:25.255008 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:25.255008 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:25.255008 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:25.256004 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:25.256004 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:25.256004 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:25.256004 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:25.256004 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 17 19:10:25.256004 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 17 19:10:25.256004 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 17 19:10:25.256004 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 17 19:10:25.256004 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 17 19:10:25.256004 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 17 19:10:25.256004 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 17 19:10:25.256004 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:10:25.256004 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 205/390] 10.222.0.58 () {56 vars in 1114 bytes} [Tue May 17 19:10:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 17 19:10:25.264076 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523bb477-c5a4-4619-bd65-4183a9be797b None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:25.392332 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523bb477-c5a4-4619-bd65-4183a9be797b None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:25.439461 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9532c51f-ca25-415e-ade0-836f243cce1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:25.440876 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9532c51f-ca25-415e-ade0-836f243cce1a None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:25.441334 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9532c51f-ca25-415e-ade0-836f243cce1a None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:25.441804 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9532c51f-ca25-415e-ade0-836f243cce1a None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:25.442778 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9532c51f-ca25-415e-ade0-836f243cce1a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:25.443802 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9532c51f-ca25-415e-ade0-836f243cce1a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:25.445368 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9532c51f-ca25-415e-ade0-836f243cce1a None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:25.453969 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 186/391] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 17 19:10:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:25.460532 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcf4fd7-c0f9-4303-b54f-e68ef5d3b20a None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:25.564593 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcf4fd7-c0f9-4303-b54f-e68ef5d3b20a None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:25.602792 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646a5f51-b949-4b53-9f4b-a4a8adf898e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:25.604042 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646a5f51-b949-4b53-9f4b-a4a8adf898e3 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:25.604501 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646a5f51-b949-4b53-9f4b-a4a8adf898e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:25.605008 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646a5f51-b949-4b53-9f4b-a4a8adf898e3 None admin] PATH_INFO: `/v3/groups/7dae670cfaf346a2982f211f2796a7c7` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:25.606371 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646a5f51-b949-4b53-9f4b-a4a8adf898e3 None admin] RBAC: Authorizing `identity:get_group(group_id=7dae670cfaf346a2982f211f2796a7c7)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:25.608034 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646a5f51-b949-4b53-9f4b-a4a8adf898e3 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:25.614404 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 206/392] 10.222.0.58 () {56 vars in 1180 bytes} [Tue May 17 19:10:25 2022] GET /identity/v3/groups/7dae670cfaf346a2982f211f2796a7c7 => generated 226 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:25.616033 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9532c51f-ca25-415e-ade0-836f243cce1a None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:25.727701 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9532c51f-ca25-415e-ade0-836f243cce1a None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:25.762623 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf987e23-13eb-470b-9472-30018cd6b382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:25.763835 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf987e23-13eb-470b-9472-30018cd6b382 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:25.764291 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf987e23-13eb-470b-9472-30018cd6b382 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:25.764735 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf987e23-13eb-470b-9472-30018cd6b382 None admin] PATH_INFO: `/v3/projects/d13e273646b44ecca3cd10cd867bbd04` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:25.766093 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf987e23-13eb-470b-9472-30018cd6b382 None admin] RBAC: Authorizing `identity:get_project(project_id=d13e273646b44ecca3cd10cd867bbd04)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:25.767758 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf987e23-13eb-470b-9472-30018cd6b382 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:25.773982 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 187/393] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 17 19:10:25 2022] GET /identity/v3/projects/d13e273646b44ecca3cd10cd867bbd04 => generated 298 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:25.775955 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646a5f51-b949-4b53-9f4b-a4a8adf898e3 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:25.906221 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646a5f51-b949-4b53-9f4b-a4a8adf898e3 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:25.948913 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed311b4e-b1e4-4081-8bff-06e80b285b4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:25.950161 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed311b4e-b1e4-4081-8bff-06e80b285b4f None admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:25.950617 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed311b4e-b1e4-4081-8bff-06e80b285b4f None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:25.951111 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed311b4e-b1e4-4081-8bff-06e80b285b4f None admin] PATH_INFO: `/v3/projects/d13e273646b44ecca3cd10cd867bbd04/groups/7dae670cfaf346a2982f211f2796a7c7/roles/1055a3235b0143e58f1090c1e154e5c1` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:25.957767 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed311b4e-b1e4-4081-8bff-06e80b285b4f None admin] RBAC: Authorizing `identity:create_grant(project_id=d13e273646b44ecca3cd10cd867bbd04, group_id=7dae670cfaf346a2982f211f2796a7c7, role_id=1055a3235b0143e58f1090c1e154e5c1)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:25.959653 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed311b4e-b1e4-4081-8bff-06e80b285b4f None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:26.030446 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 207/394] 10.222.0.58 () {58 vars in 1442 bytes} [Tue May 17 19:10:25 2022] PUT /identity/v3/projects/d13e273646b44ecca3cd10cd867bbd04/groups/7dae670cfaf346a2982f211f2796a7c7/roles/1055a3235b0143e58f1090c1e154e5c1 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 17 19:10:26.157032 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f43bf9b-52cd-49a9-b84b-717654acddeb None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:26.157675 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f43bf9b-52cd-49a9-b84b-717654acddeb None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:26.158151 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f43bf9b-52cd-49a9-b84b-717654acddeb None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:26.160220 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 188/395] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:10:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:10:26.165592 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb090283-8799-4896-9f75-bf8355df3669 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:26.165880 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb090283-8799-4896-9f75-bf8355df3669 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:26.166157 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb090283-8799-4896-9f75-bf8355df3669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:26.192624 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-cb090283-8799-4896-9f75-bf8355df3669 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:10:26.194364 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-cb090283-8799-4896-9f75-bf8355df3669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:10:26.278818 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 208/396] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:10:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 17 19:10:26.300215 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f43bf9b-52cd-49a9-b84b-717654acddeb None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:26.422636 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f43bf9b-52cd-49a9-b84b-717654acddeb None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:26.462805 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f744438a-f6da-4d74-ae2d-cb2a8015c8a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:26.464455 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f744438a-f6da-4d74-ae2d-cb2a8015c8a7 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:26.464998 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f744438a-f6da-4d74-ae2d-cb2a8015c8a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:26.465470 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f744438a-f6da-4d74-ae2d-cb2a8015c8a7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:26.480395 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f744438a-f6da-4d74-ae2d-cb2a8015c8a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 17 19:10:26.488559 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-f744438a-f6da-4d74-ae2d-cb2a8015c8a7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 17 19:10:26.488559 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:26.488559 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:26.488559 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:26.488559 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:26.488559 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:26.488559 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:26.488559 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:26.488559 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:26.488559 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:26.488559 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:26.488559 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:26.488559 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 17 19:10:26.488559 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) May 17 19:10:26.488559 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 17 19:10:26.488559 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 17 19:10:26.489428 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 17 19:10:26.489428 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 17 19:10:26.489428 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 17 19:10:26.489428 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 17 19:10:26.489428 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:26.489428 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:26.489428 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:26.489428 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:26.489428 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:26.489428 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:26.489428 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:26.489428 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:10:26.489428 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:26.489428 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:26.489428 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:26.489428 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:26.490755 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:26.490755 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:26.490755 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:26.490755 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:26.490755 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 17 19:10:26.490755 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 17 19:10:26.490755 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 17 19:10:26.490755 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 17 19:10:26.490755 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 17 19:10:26.490755 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 17 19:10:26.490755 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 17 19:10:26.490755 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:10:26.495736 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 189/397] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 17 19:10:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:10:26.498791 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb090283-8799-4896-9f75-bf8355df3669 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:26.505217 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778787fd-5564-46b5-bd54-1a20156b5c53 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:26.506327 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778787fd-5564-46b5-bd54-1a20156b5c53 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:26.506836 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778787fd-5564-46b5-bd54-1a20156b5c53 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:26.508823 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 190/398] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:10:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 17 19:10:26.514688 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e419e6cd-4a43-442a-b82e-8845791ef51e None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:26.514973 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e419e6cd-4a43-442a-b82e-8845791ef51e None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:26.515251 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e419e6cd-4a43-442a-b82e-8845791ef51e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:26.533966 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-e419e6cd-4a43-442a-b82e-8845791ef51e None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:10:26.535204 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-e419e6cd-4a43-442a-b82e-8845791ef51e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:10:26.605044 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 191/399] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:10:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 17 19:10:26.628153 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e419e6cd-4a43-442a-b82e-8845791ef51e None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:26.671249 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb090283-8799-4896-9f75-bf8355df3669 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:26.717308 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8289f81a-f4c5-481b-b7f4-4c542f44a2e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:26.718079 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8289f81a-f4c5-481b-b7f4-4c542f44a2e6 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:26.718360 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8289f81a-f4c5-481b-b7f4-4c542f44a2e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:26.718633 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8289f81a-f4c5-481b-b7f4-4c542f44a2e6 None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:26.719392 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8289f81a-f4c5-481b-b7f4-4c542f44a2e6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:26.719733 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8289f81a-f4c5-481b-b7f4-4c542f44a2e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:26.721109 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8289f81a-f4c5-481b-b7f4-4c542f44a2e6 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:26.730054 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e419e6cd-4a43-442a-b82e-8845791ef51e None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:26.730375 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 209/400] 10.222.0.58 () {56 vars in 1101 bytes} [Tue May 17 19:10:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:10:26.738775 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8289f81a-f4c5-481b-b7f4-4c542f44a2e6 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:26.784657 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edad116-c828-4100-bd80-d61f90f46a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:26.785472 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edad116-c828-4100-bd80-d61f90f46a77 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:26.785745 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edad116-c828-4100-bd80-d61f90f46a77 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:26.786014 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edad116-c828-4100-bd80-d61f90f46a77 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:26.800553 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edad116-c828-4100-bd80-d61f90f46a77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 17 19:10:26.809354 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-6edad116-c828-4100-bd80-d61f90f46a77 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 17 19:10:26.809354 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:26.809354 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:26.809354 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:26.809354 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:26.809354 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:26.809354 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:26.809354 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:26.809354 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:26.809354 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:26.809354 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:26.809354 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:26.809354 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 17 19:10:26.809354 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) May 17 19:10:26.809354 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 17 19:10:26.809354 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 17 19:10:26.810246 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 17 19:10:26.810246 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 17 19:10:26.810246 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 17 19:10:26.810246 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 17 19:10:26.810246 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:26.810246 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:26.810246 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:26.810246 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:26.810246 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:26.810246 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:26.810246 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:26.810246 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:10:26.810246 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:26.810246 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:26.810246 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:26.810246 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:26.811922 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:26.811922 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:26.811922 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:26.811922 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:26.811922 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 17 19:10:26.811922 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 17 19:10:26.811922 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 17 19:10:26.811922 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 17 19:10:26.811922 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 17 19:10:26.811922 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 17 19:10:26.811922 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 17 19:10:26.811922 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:10:26.811922 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 192/401] 10.222.0.58 () {56 vars in 1114 bytes} [Tue May 17 19:10:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 17 19:10:26.816575 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edad116-c828-4100-bd80-d61f90f46a77 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:26.867610 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8289f81a-f4c5-481b-b7f4-4c542f44a2e6 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:26.906793 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e355338a-b588-40ad-8b09-8a23133800ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:26.908266 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e355338a-b588-40ad-8b09-8a23133800ef None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:26.908548 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e355338a-b588-40ad-8b09-8a23133800ef None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:26.908818 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e355338a-b588-40ad-8b09-8a23133800ef None admin] PATH_INFO: `/v3/groups/7dae670cfaf346a2982f211f2796a7c7` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:26.910142 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e355338a-b588-40ad-8b09-8a23133800ef None admin] RBAC: Authorizing `identity:get_group(group_id=7dae670cfaf346a2982f211f2796a7c7)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:26.912806 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e355338a-b588-40ad-8b09-8a23133800ef None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:26.915170 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 210/402] 10.222.0.58 () {56 vars in 1180 bytes} [Tue May 17 19:10:26 2022] GET /identity/v3/groups/7dae670cfaf346a2982f211f2796a7c7 => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:26.920735 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e355338a-b588-40ad-8b09-8a23133800ef None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:26.937038 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edad116-c828-4100-bd80-d61f90f46a77 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:27.010361 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38daf1d-4447-49c0-9442-cae0a521175a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:27.011048 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38daf1d-4447-49c0-9442-cae0a521175a None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:27.011331 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38daf1d-4447-49c0-9442-cae0a521175a None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:27.011637 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38daf1d-4447-49c0-9442-cae0a521175a None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:27.012387 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38daf1d-4447-49c0-9442-cae0a521175a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:27.012691 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38daf1d-4447-49c0-9442-cae0a521175a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:27.014085 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38daf1d-4447-49c0-9442-cae0a521175a None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:27.032091 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e355338a-b588-40ad-8b09-8a23133800ef None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:27.034906 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 193/403] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 17 19:10:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:10:27.040837 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38daf1d-4447-49c0-9442-cae0a521175a None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:27.069385 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b980a973-a8e3-47ff-9b12-299a34ae9fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:27.070254 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b980a973-a8e3-47ff-9b12-299a34ae9fe3 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:27.070525 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b980a973-a8e3-47ff-9b12-299a34ae9fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:27.070791 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b980a973-a8e3-47ff-9b12-299a34ae9fe3 None admin] PATH_INFO: `/v3/projects/625d29bdd4ad47ee92370dc3404fe414` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:27.072168 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b980a973-a8e3-47ff-9b12-299a34ae9fe3 None admin] RBAC: Authorizing `identity:get_project(project_id=625d29bdd4ad47ee92370dc3404fe414)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:27.073785 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b980a973-a8e3-47ff-9b12-299a34ae9fe3 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:27.080326 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 211/404] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 17 19:10:26 2022] GET /identity/v3/projects/625d29bdd4ad47ee92370dc3404fe414 => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:27.081888 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b980a973-a8e3-47ff-9b12-299a34ae9fe3 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:27.189740 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b980a973-a8e3-47ff-9b12-299a34ae9fe3 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:27.229700 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d593b5-148e-44cd-bdfa-61478e6a6aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:27.230746 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d593b5-148e-44cd-bdfa-61478e6a6aad None admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:27.231025 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d593b5-148e-44cd-bdfa-61478e6a6aad None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:27.231291 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d593b5-148e-44cd-bdfa-61478e6a6aad None admin] PATH_INFO: `/v3/projects/625d29bdd4ad47ee92370dc3404fe414/groups/7dae670cfaf346a2982f211f2796a7c7/roles/114e6a16e63447d1924707ace5aace25` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:27.233315 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d593b5-148e-44cd-bdfa-61478e6a6aad None admin] RBAC: Authorizing `identity:create_grant(project_id=625d29bdd4ad47ee92370dc3404fe414, group_id=7dae670cfaf346a2982f211f2796a7c7, role_id=114e6a16e63447d1924707ace5aace25)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:27.234741 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d593b5-148e-44cd-bdfa-61478e6a6aad None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:27.277193 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38daf1d-4447-49c0-9442-cae0a521175a None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:27.298940 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 212/405] 10.222.0.58 () {58 vars in 1442 bytes} [Tue May 17 19:10:27 2022] PUT /identity/v3/projects/625d29bdd4ad47ee92370dc3404fe414/groups/7dae670cfaf346a2982f211f2796a7c7/roles/114e6a16e63447d1924707ace5aace25 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:10:27.331150 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ff80b9-48f9-4cf9-bc5b-8ffe9d33d3e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:27.332059 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ff80b9-48f9-4cf9-bc5b-8ffe9d33d3e5 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:27.332338 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ff80b9-48f9-4cf9-bc5b-8ffe9d33d3e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:27.332603 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ff80b9-48f9-4cf9-bc5b-8ffe9d33d3e5 None admin] PATH_INFO: `/v3/groups/7dae670cfaf346a2982f211f2796a7c7` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:27.334285 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ff80b9-48f9-4cf9-bc5b-8ffe9d33d3e5 None admin] RBAC: Authorizing `identity:get_group(group_id=7dae670cfaf346a2982f211f2796a7c7)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:27.335759 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ff80b9-48f9-4cf9-bc5b-8ffe9d33d3e5 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:27.337379 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 194/406] 10.222.0.58 () {56 vars in 1180 bytes} [Tue May 17 19:10:27 2022] GET /identity/v3/groups/7dae670cfaf346a2982f211f2796a7c7 => generated 226 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:27.343084 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d593b5-148e-44cd-bdfa-61478e6a6aad None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:27.408217 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e78e71-ef97-46a6-8bc3-f686c9b0b654 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:27.408529 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e78e71-ef97-46a6-8bc3-f686c9b0b654 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:27.408746 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e78e71-ef97-46a6-8bc3-f686c9b0b654 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:27.410127 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 195/407] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:10:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 17 19:10:27.416431 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57209987-42fe-4b3c-b5c9-9d1ad5c92563 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:27.417080 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57209987-42fe-4b3c-b5c9-9d1ad5c92563 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:27.417455 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57209987-42fe-4b3c-b5c9-9d1ad5c92563 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:27.453070 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-57209987-42fe-4b3c-b5c9-9d1ad5c92563 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:10:27.453995 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-57209987-42fe-4b3c-b5c9-9d1ad5c92563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:10:27.465491 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d593b5-148e-44cd-bdfa-61478e6a6aad None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:27.506205 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad69c4e0-5ecd-43aa-8f86-b0c6634b18a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:27.507803 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad69c4e0-5ecd-43aa-8f86-b0c6634b18a2 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:27.508229 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad69c4e0-5ecd-43aa-8f86-b0c6634b18a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:27.508823 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad69c4e0-5ecd-43aa-8f86-b0c6634b18a2 None admin] PATH_INFO: `/v3/projects/625d29bdd4ad47ee92370dc3404fe414` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:27.510586 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad69c4e0-5ecd-43aa-8f86-b0c6634b18a2 None admin] RBAC: Authorizing `identity:get_project(project_id=625d29bdd4ad47ee92370dc3404fe414)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:27.512717 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad69c4e0-5ecd-43aa-8f86-b0c6634b18a2 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:27.515536 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 213/408] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 17 19:10:27 2022] GET /identity/v3/projects/625d29bdd4ad47ee92370dc3404fe414 => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:27.525853 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad69c4e0-5ecd-43aa-8f86-b0c6634b18a2 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:27.581768 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 196/409] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:10:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 17 19:10:27.606619 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57209987-42fe-4b3c-b5c9-9d1ad5c92563 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:27.634487 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad69c4e0-5ecd-43aa-8f86-b0c6634b18a2 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:27.669773 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f529a4a0-4b2f-44d7-9728-aef3e0f108ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:27.670653 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f529a4a0-4b2f-44d7-9728-aef3e0f108ba None admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:27.671776 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f529a4a0-4b2f-44d7-9728-aef3e0f108ba None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:27.671776 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f529a4a0-4b2f-44d7-9728-aef3e0f108ba None admin] PATH_INFO: `/v3/projects/625d29bdd4ad47ee92370dc3404fe414/groups/7dae670cfaf346a2982f211f2796a7c7/roles/1055a3235b0143e58f1090c1e154e5c1` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:27.674976 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f529a4a0-4b2f-44d7-9728-aef3e0f108ba None admin] RBAC: Authorizing `identity:create_grant(project_id=625d29bdd4ad47ee92370dc3404fe414, group_id=7dae670cfaf346a2982f211f2796a7c7, role_id=1055a3235b0143e58f1090c1e154e5c1)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:27.676473 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f529a4a0-4b2f-44d7-9728-aef3e0f108ba None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:27.726640 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57209987-42fe-4b3c-b5c9-9d1ad5c92563 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:27.761317 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd2ec69-b4be-421a-b0b9-1e8d6bb1ea08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:27.762552 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd2ec69-b4be-421a-b0b9-1e8d6bb1ea08 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:27.763020 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd2ec69-b4be-421a-b0b9-1e8d6bb1ea08 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:27.763476 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd2ec69-b4be-421a-b0b9-1e8d6bb1ea08 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:27.774004 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd2ec69-b4be-421a-b0b9-1e8d6bb1ea08 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 17 19:10:27.781776 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-cfd2ec69-b4be-421a-b0b9-1e8d6bb1ea08 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 17 19:10:27.781776 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:27.781776 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:27.781776 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:27.781776 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:27.781776 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:27.781776 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:27.781776 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:27.781776 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:27.781776 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:27.781776 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:27.781776 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:27.781776 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 17 19:10:27.781776 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) May 17 19:10:27.781776 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 17 19:10:27.781776 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 17 19:10:27.782636 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 17 19:10:27.782636 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 17 19:10:27.782636 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 17 19:10:27.782636 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 17 19:10:27.782636 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:27.782636 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:27.782636 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:27.782636 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:27.782636 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:27.782636 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:27.782636 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:27.782636 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:10:27.782636 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:27.782636 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:27.782636 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:27.782636 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:27.783415 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:27.783415 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:27.783415 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:27.783415 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:27.783415 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 17 19:10:27.783415 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 17 19:10:27.783415 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 17 19:10:27.783415 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 17 19:10:27.783415 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 17 19:10:27.783415 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 17 19:10:27.783415 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 17 19:10:27.783415 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:10:27.784583 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 214/410] 10.222.0.58 () {58 vars in 1442 bytes} [Tue May 17 19:10:27 2022] PUT /identity/v3/projects/625d29bdd4ad47ee92370dc3404fe414/groups/7dae670cfaf346a2982f211f2796a7c7/roles/1055a3235b0143e58f1090c1e154e5c1 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 17 19:10:27.786325 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 197/411] 10.222.0.58 () {56 vars in 1096 bytes} [Tue May 17 19:10:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 17 19:10:27.792837 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f529a4a0-4b2f-44d7-9728-aef3e0f108ba None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:27.909818 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f529a4a0-4b2f-44d7-9728-aef3e0f108ba None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:27.956516 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8574fa-3dcf-4b19-83b0-9770d81ac80d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:27.957564 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8574fa-3dcf-4b19-83b0-9770d81ac80d None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:27.958023 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8574fa-3dcf-4b19-83b0-9770d81ac80d None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:27.958475 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8574fa-3dcf-4b19-83b0-9770d81ac80d None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:27.960014 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8574fa-3dcf-4b19-83b0-9770d81ac80d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:27.960527 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8574fa-3dcf-4b19-83b0-9770d81ac80d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:27.962152 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8574fa-3dcf-4b19-83b0-9770d81ac80d None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:27.972217 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 215/412] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 17 19:10:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:27.978921 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd2ec69-b4be-421a-b0b9-1e8d6bb1ea08 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:28.095971 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd2ec69-b4be-421a-b0b9-1e8d6bb1ea08 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:28.131881 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98608647-a288-4ae5-8f66-2170c499cee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:28.133025 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98608647-a288-4ae5-8f66-2170c499cee1 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:28.133474 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98608647-a288-4ae5-8f66-2170c499cee1 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:28.133912 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98608647-a288-4ae5-8f66-2170c499cee1 None admin] PATH_INFO: `/v3/groups/87c9fe75443848f894d3e8f7b86160ca` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:28.135324 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98608647-a288-4ae5-8f66-2170c499cee1 None admin] RBAC: Authorizing `identity:get_group(group_id=87c9fe75443848f894d3e8f7b86160ca)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:28.137039 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98608647-a288-4ae5-8f66-2170c499cee1 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:28.139108 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 198/413] 10.222.0.58 () {56 vars in 1180 bytes} [Tue May 17 19:10:27 2022] GET /identity/v3/groups/87c9fe75443848f894d3e8f7b86160ca => generated 229 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:28.144601 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8574fa-3dcf-4b19-83b0-9770d81ac80d None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:28.254439 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8574fa-3dcf-4b19-83b0-9770d81ac80d None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:28.288941 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb00100-6f9a-474a-91d4-a85ffb014558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:28.290173 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb00100-6f9a-474a-91d4-a85ffb014558 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:28.290665 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb00100-6f9a-474a-91d4-a85ffb014558 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:28.291104 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb00100-6f9a-474a-91d4-a85ffb014558 None admin] PATH_INFO: `/v3/projects/07bd98f7df3a477a8bf75754083485f0` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:28.292479 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb00100-6f9a-474a-91d4-a85ffb014558 None admin] RBAC: Authorizing `identity:get_project(project_id=07bd98f7df3a477a8bf75754083485f0)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:28.294099 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb00100-6f9a-474a-91d4-a85ffb014558 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:28.296268 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 216/414] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 17 19:10:28 2022] GET /identity/v3/projects/07bd98f7df3a477a8bf75754083485f0 => generated 344 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:28.302105 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98608647-a288-4ae5-8f66-2170c499cee1 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:28.416713 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98608647-a288-4ae5-8f66-2170c499cee1 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:28.454340 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f769e840-679a-4c7a-8215-929a786b33e0 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:28.454989 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f769e840-679a-4c7a-8215-929a786b33e0 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:28.455454 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f769e840-679a-4c7a-8215-929a786b33e0 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:28.457278 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 217/415] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:10:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 17 19:10:28.459517 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb93446-e755-4f93-8092-ac1976584ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:28.461902 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb93446-e755-4f93-8092-ac1976584ec8 None admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:28.461902 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb93446-e755-4f93-8092-ac1976584ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:28.461902 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb93446-e755-4f93-8092-ac1976584ec8 None admin] PATH_INFO: `/v3/projects/07bd98f7df3a477a8bf75754083485f0/groups/87c9fe75443848f894d3e8f7b86160ca/roles/60af5088793e4379b88821a86552a344` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:28.463545 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3166a996-0627-404c-85bd-e099f3896df9 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:28.464029 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3166a996-0627-404c-85bd-e099f3896df9 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:28.464492 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3166a996-0627-404c-85bd-e099f3896df9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:28.465629 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb93446-e755-4f93-8092-ac1976584ec8 None admin] RBAC: Authorizing `identity:create_grant(project_id=07bd98f7df3a477a8bf75754083485f0, group_id=87c9fe75443848f894d3e8f7b86160ca, role_id=60af5088793e4379b88821a86552a344)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:28.467381 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb93446-e755-4f93-8092-ac1976584ec8 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:28.484865 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-3166a996-0627-404c-85bd-e099f3896df9 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:10:28.486355 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-3166a996-0627-404c-85bd-e099f3896df9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:10:28.556710 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 218/416] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:10:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 17 19:10:28.580844 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3166a996-0627-404c-85bd-e099f3896df9 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:28.601925 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 199/417] 10.222.0.58 () {58 vars in 1442 bytes} [Tue May 17 19:10:28 2022] PUT /identity/v3/projects/07bd98f7df3a477a8bf75754083485f0/groups/87c9fe75443848f894d3e8f7b86160ca/roles/60af5088793e4379b88821a86552a344 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:10:28.716620 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3166a996-0627-404c-85bd-e099f3896df9 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:28.767151 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070a89ec-eb5c-49ab-97c9-6ad7fa7ff24f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:28.767997 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070a89ec-eb5c-49ab-97c9-6ad7fa7ff24f None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:28.768277 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070a89ec-eb5c-49ab-97c9-6ad7fa7ff24f None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:28.768548 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070a89ec-eb5c-49ab-97c9-6ad7fa7ff24f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:28.779980 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070a89ec-eb5c-49ab-97c9-6ad7fa7ff24f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 17 19:10:28.787988 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-070a89ec-eb5c-49ab-97c9-6ad7fa7ff24f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 17 19:10:28.787988 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:28.787988 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:28.787988 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:28.787988 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:28.787988 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:28.787988 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:28.787988 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:28.787988 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:28.787988 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:28.787988 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:28.787988 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:28.787988 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 17 19:10:28.787988 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) May 17 19:10:28.787988 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 17 19:10:28.787988 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 17 19:10:28.788934 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 17 19:10:28.788934 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 17 19:10:28.788934 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 17 19:10:28.788934 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 17 19:10:28.788934 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:28.788934 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:28.788934 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:28.788934 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:28.788934 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:28.788934 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:28.788934 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:28.788934 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:10:28.788934 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:28.788934 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:28.788934 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:28.788934 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:28.789696 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:28.789696 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:28.789696 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:28.789696 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:28.789696 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 17 19:10:28.789696 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 17 19:10:28.789696 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 17 19:10:28.789696 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 17 19:10:28.789696 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 17 19:10:28.789696 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 17 19:10:28.789696 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 17 19:10:28.789696 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:10:28.789696 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 219/418] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 17 19:10:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 17 19:10:28.796605 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb93446-e755-4f93-8092-ac1976584ec8 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:28.932451 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb93446-e755-4f93-8092-ac1976584ec8 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:28.967226 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe14fd46-7002-4f93-a52f-b0e4e14a5a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:28.967926 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe14fd46-7002-4f93-a52f-b0e4e14a5a7e None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:28.968201 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe14fd46-7002-4f93-a52f-b0e4e14a5a7e None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:28.968468 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe14fd46-7002-4f93-a52f-b0e4e14a5a7e None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:28.969202 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe14fd46-7002-4f93-a52f-b0e4e14a5a7e None admin] RBAC: Adding query filter params (name=member) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:28.969499 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe14fd46-7002-4f93-a52f-b0e4e14a5a7e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:28.970861 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe14fd46-7002-4f93-a52f-b0e4e14a5a7e None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:28.979674 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 200/419] 10.222.0.58 () {56 vars in 1101 bytes} [Tue May 17 19:10:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:28.985000 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070a89ec-eb5c-49ab-97c9-6ad7fa7ff24f None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:29.129457 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070a89ec-eb5c-49ab-97c9-6ad7fa7ff24f None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:29.181089 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e4f865-f249-4838-9bd6-2a0d61552046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:29.181739 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e4f865-f249-4838-9bd6-2a0d61552046 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:29.182016 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e4f865-f249-4838-9bd6-2a0d61552046 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:29.182283 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e4f865-f249-4838-9bd6-2a0d61552046 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:29.237090 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab202dc0-e17c-4ed4-bb33-7daa4ea3882b None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:29.237447 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab202dc0-e17c-4ed4-bb33-7daa4ea3882b None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:29.237621 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab202dc0-e17c-4ed4-bb33-7daa4ea3882b None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:29.239144 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 201/420] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:10:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:10:29.253232 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb927280-e209-4d36-9587-2d325e8ab04d None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:29.253503 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb927280-e209-4d36-9587-2d325e8ab04d None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:29.253769 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb927280-e209-4d36-9587-2d325e8ab04d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:29.274979 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-cb927280-e209-4d36-9587-2d325e8ab04d None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:10:29.276177 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-cb927280-e209-4d36-9587-2d325e8ab04d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:10:29.285604 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e4f865-f249-4838-9bd6-2a0d61552046 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:29.287788 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e4f865-f249-4838-9bd6-2a0d61552046 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:29.348097 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 202/421] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:10:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 17 19:10:29.371698 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb927280-e209-4d36-9587-2d325e8ab04d None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:29.397565 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 220/422] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 17 19:10:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 17 19:10:29.403301 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e4f865-f249-4838-9bd6-2a0d61552046 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:29.486437 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb927280-e209-4d36-9587-2d325e8ab04d None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:29.520061 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdd868a-2d0c-4de1-8956-9d340dcf9109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:29.521623 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdd868a-2d0c-4de1-8956-9d340dcf9109 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:29.522050 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdd868a-2d0c-4de1-8956-9d340dcf9109 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:29.522455 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdd868a-2d0c-4de1-8956-9d340dcf9109 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:29.532922 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdd868a-2d0c-4de1-8956-9d340dcf9109 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 17 19:10:29.541157 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-6fdd868a-2d0c-4de1-8956-9d340dcf9109 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 17 19:10:29.541157 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:29.541157 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:29.541157 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:29.541157 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:29.541157 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:29.541157 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:29.541157 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:29.541157 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:29.541157 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:29.541157 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:29.541157 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:29.541157 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 17 19:10:29.541157 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) May 17 19:10:29.541157 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 17 19:10:29.541157 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 17 19:10:29.542211 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 17 19:10:29.542211 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 17 19:10:29.542211 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 17 19:10:29.542211 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 17 19:10:29.542211 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:29.542211 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:29.542211 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:29.542211 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:29.542211 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:29.542211 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:29.542211 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:29.542211 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:10:29.542211 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:29.542211 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:29.542211 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:29.542211 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:29.543195 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:29.543195 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:29.543195 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:29.543195 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:29.543195 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 17 19:10:29.543195 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 17 19:10:29.543195 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 17 19:10:29.543195 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 17 19:10:29.543195 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 17 19:10:29.543195 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 17 19:10:29.543195 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 17 19:10:29.543195 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:10:29.545538 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 203/423] 10.222.0.58 () {56 vars in 1114 bytes} [Tue May 17 19:10:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 17 19:10:29.551150 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdd868a-2d0c-4de1-8956-9d340dcf9109 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:29.591298 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e4f865-f249-4838-9bd6-2a0d61552046 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:29.658080 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdd868a-2d0c-4de1-8956-9d340dcf9109 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:29.660167 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51e0fe6-2419-40be-ac48-c6b466716817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:29.661644 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51e0fe6-2419-40be-ac48-c6b466716817 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:29.661644 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51e0fe6-2419-40be-ac48-c6b466716817 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:29.661644 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51e0fe6-2419-40be-ac48-c6b466716817 None admin] PATH_INFO: `/v3/projects/d13e273646b44ecca3cd10cd867bbd04` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:29.662824 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51e0fe6-2419-40be-ac48-c6b466716817 None admin] RBAC: Authorizing `identity:get_project(project_id=d13e273646b44ecca3cd10cd867bbd04)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:29.664299 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51e0fe6-2419-40be-ac48-c6b466716817 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:29.666651 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 221/424] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 17 19:10:29 2022] GET /identity/v3/projects/d13e273646b44ecca3cd10cd867bbd04 => generated 298 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:10:29.671326 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51e0fe6-2419-40be-ac48-c6b466716817 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:29.694732 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0df02fa-c5df-4bbd-924b-85c578608abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:29.695700 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0df02fa-c5df-4bbd-924b-85c578608abf None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:29.696116 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0df02fa-c5df-4bbd-924b-85c578608abf None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:29.696599 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0df02fa-c5df-4bbd-924b-85c578608abf None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:29.697574 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0df02fa-c5df-4bbd-924b-85c578608abf None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:29.698009 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0df02fa-c5df-4bbd-924b-85c578608abf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:29.699645 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0df02fa-c5df-4bbd-924b-85c578608abf None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:29.708402 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 204/425] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 17 19:10:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:10:29.714043 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0df02fa-c5df-4bbd-924b-85c578608abf None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:29.813442 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0df02fa-c5df-4bbd-924b-85c578608abf None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:29.848197 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0567ac-394c-4282-b4e6-c9482318ad09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:29.849082 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0567ac-394c-4282-b4e6-c9482318ad09 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:29.850014 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0567ac-394c-4282-b4e6-c9482318ad09 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:29.850441 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0567ac-394c-4282-b4e6-c9482318ad09 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:29.914178 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51e0fe6-2419-40be-ac48-c6b466716817 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:29.922666 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0567ac-394c-4282-b4e6-c9482318ad09 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:29.924441 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0567ac-394c-4282-b4e6-c9482318ad09 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:30.004207 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9128a4a7-f66a-471e-ab66-eb61bf599da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:30.005083 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9128a4a7-f66a-471e-ab66-eb61bf599da1 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:30.005361 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9128a4a7-f66a-471e-ab66-eb61bf599da1 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:30.006166 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9128a4a7-f66a-471e-ab66-eb61bf599da1 None admin] PATH_INFO: `/v3/groups/7dae670cfaf346a2982f211f2796a7c7` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:30.012209 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9128a4a7-f66a-471e-ab66-eb61bf599da1 None admin] RBAC: Authorizing `identity:get_group(group_id=7dae670cfaf346a2982f211f2796a7c7)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:30.013680 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9128a4a7-f66a-471e-ab66-eb61bf599da1 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:30.015665 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 222/426] 10.222.0.58 () {56 vars in 1180 bytes} [Tue May 17 19:10:29 2022] GET /identity/v3/groups/7dae670cfaf346a2982f211f2796a7c7 => generated 226 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:10:30.020850 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9128a4a7-f66a-471e-ab66-eb61bf599da1 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:30.025055 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 205/427] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 17 19:10:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 17 19:10:30.030380 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0567ac-394c-4282-b4e6-c9482318ad09 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:30.128389 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0567ac-394c-4282-b4e6-c9482318ad09 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:30.162738 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd6dca7-338c-49f3-aa3c-36d935c0de50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:30.163619 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd6dca7-338c-49f3-aa3c-36d935c0de50 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:30.163892 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd6dca7-338c-49f3-aa3c-36d935c0de50 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:30.164155 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd6dca7-338c-49f3-aa3c-36d935c0de50 None admin] PATH_INFO: `/v3/projects/d13e273646b44ecca3cd10cd867bbd04` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:30.165343 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd6dca7-338c-49f3-aa3c-36d935c0de50 None admin] RBAC: Authorizing `identity:get_project(project_id=d13e273646b44ecca3cd10cd867bbd04)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:30.166771 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd6dca7-338c-49f3-aa3c-36d935c0de50 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:30.168783 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 206/428] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 17 19:10:30 2022] GET /identity/v3/projects/d13e273646b44ecca3cd10cd867bbd04 => generated 298 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:10:30.173954 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd6dca7-338c-49f3-aa3c-36d935c0de50 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:30.273839 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9128a4a7-f66a-471e-ab66-eb61bf599da1 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:30.278978 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd6dca7-338c-49f3-aa3c-36d935c0de50 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:30.313150 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de3b4e2-b4a8-4c9a-891c-d1b4feed00e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:30.314064 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de3b4e2-b4a8-4c9a-891c-d1b4feed00e2 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:30.314344 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de3b4e2-b4a8-4c9a-891c-d1b4feed00e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:30.314612 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de3b4e2-b4a8-4c9a-891c-d1b4feed00e2 None admin] PATH_INFO: `/v3/groups/7dae670cfaf346a2982f211f2796a7c7` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:30.315823 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de3b4e2-b4a8-4c9a-891c-d1b4feed00e2 None admin] RBAC: Authorizing `identity:get_group(group_id=7dae670cfaf346a2982f211f2796a7c7)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:30.317263 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de3b4e2-b4a8-4c9a-891c-d1b4feed00e2 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:30.319217 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 207/429] 10.222.0.58 () {56 vars in 1180 bytes} [Tue May 17 19:10:30 2022] GET /identity/v3/groups/7dae670cfaf346a2982f211f2796a7c7 => generated 226 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:10:30.325964 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de3b4e2-b4a8-4c9a-891c-d1b4feed00e2 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:30.351930 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706830e8-d79c-4eaf-8d79-6a404063492b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:30.352633 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706830e8-d79c-4eaf-8d79-6a404063492b None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:30.353071 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706830e8-d79c-4eaf-8d79-6a404063492b None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:30.353536 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706830e8-d79c-4eaf-8d79-6a404063492b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:30.354497 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706830e8-d79c-4eaf-8d79-6a404063492b None admin] RBAC: Adding query filter params (group.id=7dae670cfaf346a2982f211f2796a7c7, role.id=114e6a16e63447d1924707ace5aace25, scope.project.id=d13e273646b44ecca3cd10cd867bbd04) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:30.354951 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706830e8-d79c-4eaf-8d79-6a404063492b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:30.357012 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706830e8-d79c-4eaf-8d79-6a404063492b None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:30.363935 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 223/430] 10.222.0.58 () {56 vars in 1376 bytes} [Tue May 17 19:10:30 2022] GET /identity/v3/role_assignments?group.id=7dae670cfaf346a2982f211f2796a7c7&scope.project.id=d13e273646b44ecca3cd10cd867bbd04&role.id=114e6a16e63447d1924707ace5aace25 => generated 611 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:30.430691 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9174b7-49b7-4829-9b1d-db6ddc6f756e None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:30.431315 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9174b7-49b7-4829-9b1d-db6ddc6f756e None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:30.431826 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9174b7-49b7-4829-9b1d-db6ddc6f756e None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:30.433873 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 224/431] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:10:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 17 19:10:30.439005 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f026b432-14b1-470f-8d4c-e768e75545d2 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:30.439606 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f026b432-14b1-470f-8d4c-e768e75545d2 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:30.440084 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f026b432-14b1-470f-8d4c-e768e75545d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:30.455220 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de3b4e2-b4a8-4c9a-891c-d1b4feed00e2 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:30.472151 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-f026b432-14b1-470f-8d4c-e768e75545d2 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:10:30.473740 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-f026b432-14b1-470f-8d4c-e768e75545d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:10:30.500587 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baa2b77-ec81-4027-9f88-2841f1a75637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:30.501258 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baa2b77-ec81-4027-9f88-2841f1a75637 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:30.501532 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baa2b77-ec81-4027-9f88-2841f1a75637 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:30.501797 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baa2b77-ec81-4027-9f88-2841f1a75637 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:30.502631 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8baa2b77-ec81-4027-9f88-2841f1a75637 None admin] RBAC: Adding query filter params (group.id=7dae670cfaf346a2982f211f2796a7c7, role.id=1055a3235b0143e58f1090c1e154e5c1, scope.project.id=d13e273646b44ecca3cd10cd867bbd04) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:30.502935 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8baa2b77-ec81-4027-9f88-2841f1a75637 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:30.505549 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8baa2b77-ec81-4027-9f88-2841f1a75637 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:30.518448 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 208/432] 10.222.0.58 () {56 vars in 1376 bytes} [Tue May 17 19:10:30 2022] GET /identity/v3/role_assignments?group.id=7dae670cfaf346a2982f211f2796a7c7&scope.project.id=d13e273646b44ecca3cd10cd867bbd04&role.id=1055a3235b0143e58f1090c1e154e5c1 => generated 611 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:30.579323 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 225/433] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:10:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 17 19:10:30.605059 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baa2b77-ec81-4027-9f88-2841f1a75637 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:30.727860 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baa2b77-ec81-4027-9f88-2841f1a75637 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:30.769785 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8189b3d4-8b11-442d-a939-9c45071145a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:30.770597 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8189b3d4-8b11-442d-a939-9c45071145a1 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:30.770869 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8189b3d4-8b11-442d-a939-9c45071145a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:30.771839 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8189b3d4-8b11-442d-a939-9c45071145a1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:30.788184 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8189b3d4-8b11-442d-a939-9c45071145a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 17 19:10:30.795935 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-8189b3d4-8b11-442d-a939-9c45071145a1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 17 19:10:30.795935 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:30.795935 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:30.795935 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:30.795935 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:30.795935 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:30.795935 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:30.795935 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:30.795935 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:30.795935 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:30.795935 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:30.795935 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:30.795935 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 17 19:10:30.795935 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) May 17 19:10:30.795935 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 17 19:10:30.795935 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 17 19:10:30.796833 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 17 19:10:30.796833 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 17 19:10:30.796833 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 17 19:10:30.796833 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 17 19:10:30.796833 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:30.796833 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:30.796833 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:30.796833 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:30.796833 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:30.796833 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:30.796833 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:30.796833 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:10:30.796833 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:30.796833 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:30.796833 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:30.796833 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:30.797615 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:30.797615 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:30.797615 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:30.797615 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:30.797615 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 17 19:10:30.797615 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 17 19:10:30.797615 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 17 19:10:30.797615 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 17 19:10:30.797615 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 17 19:10:30.797615 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 17 19:10:30.797615 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 17 19:10:30.797615 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:10:30.797615 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 209/434] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 17 19:10:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:10:30.804116 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f026b432-14b1-470f-8d4c-e768e75545d2 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:30.834123 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e224ba-99ba-44c7-8e69-0a2008d7a28d None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:30.834778 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e224ba-99ba-44c7-8e69-0a2008d7a28d None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:30.835257 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e224ba-99ba-44c7-8e69-0a2008d7a28d None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:30.837108 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 210/435] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:10:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:10:30.842666 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db62fd6-6898-4470-b331-d6881005285f None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:30.843213 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db62fd6-6898-4470-b331-d6881005285f None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:30.843733 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db62fd6-6898-4470-b331-d6881005285f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:30.864046 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-0db62fd6-6898-4470-b331-d6881005285f None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:10:30.865737 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-0db62fd6-6898-4470-b331-d6881005285f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:10:30.924493 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f026b432-14b1-470f-8d4c-e768e75545d2 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:30.951712 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 211/436] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:10:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 17 19:10:30.973457 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8669a765-ca7a-4715-b754-433163c4d86a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:30.974251 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8669a765-ca7a-4715-b754-433163c4d86a None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:30.974526 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8669a765-ca7a-4715-b754-433163c4d86a None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:30.975136 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8669a765-ca7a-4715-b754-433163c4d86a None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:30.975918 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8669a765-ca7a-4715-b754-433163c4d86a None admin] RBAC: Adding query filter params (name=member) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:30.976216 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8669a765-ca7a-4715-b754-433163c4d86a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:30.976487 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db62fd6-6898-4470-b331-d6881005285f None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:30.977581 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8669a765-ca7a-4715-b754-433163c4d86a None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:30.994277 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 226/437] 10.222.0.58 () {56 vars in 1101 bytes} [Tue May 17 19:10:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:30.999964 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8669a765-ca7a-4715-b754-433163c4d86a None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:31.097360 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db62fd6-6898-4470-b331-d6881005285f None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:31.125412 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8669a765-ca7a-4715-b754-433163c4d86a None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:31.139943 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da98250e-4e69-40f5-b366-c9f9bdcdb53b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:31.141206 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da98250e-4e69-40f5-b366-c9f9bdcdb53b None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:31.141529 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da98250e-4e69-40f5-b366-c9f9bdcdb53b None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:31.141808 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da98250e-4e69-40f5-b366-c9f9bdcdb53b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:31.153859 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da98250e-4e69-40f5-b366-c9f9bdcdb53b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 17 19:10:31.162441 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-da98250e-4e69-40f5-b366-c9f9bdcdb53b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 17 19:10:31.162441 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:31.162441 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:31.162441 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:31.162441 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:31.162441 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:31.162441 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:31.162441 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:31.162441 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:31.162441 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:31.162441 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:31.162441 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:31.162441 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 17 19:10:31.162441 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) May 17 19:10:31.162441 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 17 19:10:31.162441 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 17 19:10:31.163364 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 17 19:10:31.163364 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 17 19:10:31.163364 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 17 19:10:31.163364 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 17 19:10:31.163364 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:31.163364 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:31.163364 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:31.163364 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:31.163364 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:31.163364 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:31.163364 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:31.163364 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:10:31.163364 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:31.163364 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:31.163364 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:31.163364 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:31.164208 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:31.164208 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:31.164208 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:31.164208 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:31.164208 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 17 19:10:31.164208 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 17 19:10:31.164208 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 17 19:10:31.164208 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 17 19:10:31.164208 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 17 19:10:31.164208 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 17 19:10:31.164208 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 17 19:10:31.164208 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:10:31.164208 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 212/438] 10.222.0.58 () {56 vars in 1114 bytes} [Tue May 17 19:10:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 17 19:10:31.171687 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da98250e-4e69-40f5-b366-c9f9bdcdb53b None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:31.179274 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3722cea-70ce-4c30-8978-21645ec08c99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:31.179959 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3722cea-70ce-4c30-8978-21645ec08c99 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:31.180238 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3722cea-70ce-4c30-8978-21645ec08c99 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:31.180507 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3722cea-70ce-4c30-8978-21645ec08c99 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:31.262745 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3722cea-70ce-4c30-8978-21645ec08c99 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:31.264651 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3722cea-70ce-4c30-8978-21645ec08c99 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:31.297079 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da98250e-4e69-40f5-b366-c9f9bdcdb53b None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:31.332093 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efeaf5b-c96e-485a-8a17-e33ba1c48ae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:31.332757 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efeaf5b-c96e-485a-8a17-e33ba1c48ae4 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:31.333033 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efeaf5b-c96e-485a-8a17-e33ba1c48ae4 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:31.333299 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efeaf5b-c96e-485a-8a17-e33ba1c48ae4 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:31.334061 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efeaf5b-c96e-485a-8a17-e33ba1c48ae4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:31.334358 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efeaf5b-c96e-485a-8a17-e33ba1c48ae4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:31.335743 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efeaf5b-c96e-485a-8a17-e33ba1c48ae4 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:31.343612 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 213/439] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 17 19:10:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:10:31.349673 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efeaf5b-c96e-485a-8a17-e33ba1c48ae4 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:31.376108 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 227/440] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 17 19:10:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 17 19:10:31.381485 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3722cea-70ce-4c30-8978-21645ec08c99 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:31.450574 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efeaf5b-c96e-485a-8a17-e33ba1c48ae4 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:31.475925 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3722cea-70ce-4c30-8978-21645ec08c99 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:31.482435 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba1903c-37c8-42e3-9c95-6bda6cbd3d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:31.483080 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba1903c-37c8-42e3-9c95-6bda6cbd3d3e None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:31.483358 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba1903c-37c8-42e3-9c95-6bda6cbd3d3e None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:31.483669 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba1903c-37c8-42e3-9c95-6bda6cbd3d3e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:31.507350 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad506635-1582-4e83-aa96-5f9812375919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:31.508238 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad506635-1582-4e83-aa96-5f9812375919 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:31.508510 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad506635-1582-4e83-aa96-5f9812375919 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:31.508772 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad506635-1582-4e83-aa96-5f9812375919 None admin] PATH_INFO: `/v3/projects/625d29bdd4ad47ee92370dc3404fe414` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:31.509928 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad506635-1582-4e83-aa96-5f9812375919 None admin] RBAC: Authorizing `identity:get_project(project_id=625d29bdd4ad47ee92370dc3404fe414)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:31.511346 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad506635-1582-4e83-aa96-5f9812375919 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:31.513096 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 228/441] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 17 19:10:31 2022] GET /identity/v3/projects/625d29bdd4ad47ee92370dc3404fe414 => generated 302 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:31.514172 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b977cad-b3e1-4f68-abac-c11b1ef93371 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:31.514448 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b977cad-b3e1-4f68-abac-c11b1ef93371 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:31.514715 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b977cad-b3e1-4f68-abac-c11b1ef93371 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:31.516084 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 229/442] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:10:31 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:10:31.520435 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b977cad-b3e1-4f68-abac-c11b1ef93371 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:31.548154 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba1903c-37c8-42e3-9c95-6bda6cbd3d3e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:31.549612 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba1903c-37c8-42e3-9c95-6bda6cbd3d3e None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:31.614781 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b977cad-b3e1-4f68-abac-c11b1ef93371 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:31.641766 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 214/443] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 17 19:10:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 17 19:10:31.641766 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c850b41-67bf-4ed7-8054-ad55b2d3a073 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:31.641766 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c850b41-67bf-4ed7-8054-ad55b2d3a073 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:31.641766 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c850b41-67bf-4ed7-8054-ad55b2d3a073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:31.647749 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6ca8dd-e7db-483a-981d-c143835dfdb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:31.648596 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6ca8dd-e7db-483a-981d-c143835dfdb5 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:31.648872 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6ca8dd-e7db-483a-981d-c143835dfdb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:31.649146 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6ca8dd-e7db-483a-981d-c143835dfdb5 None admin] PATH_INFO: `/v3/groups/7dae670cfaf346a2982f211f2796a7c7` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:31.650334 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6ca8dd-e7db-483a-981d-c143835dfdb5 None admin] RBAC: Authorizing `identity:get_group(group_id=7dae670cfaf346a2982f211f2796a7c7)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:31.651803 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6ca8dd-e7db-483a-981d-c143835dfdb5 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:31.653422 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 230/444] 10.222.0.58 () {56 vars in 1180 bytes} [Tue May 17 19:10:31 2022] GET /identity/v3/groups/7dae670cfaf346a2982f211f2796a7c7 => generated 226 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:31.654180 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6ca8dd-e7db-483a-981d-c143835dfdb5 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:31.660988 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-0c850b41-67bf-4ed7-8054-ad55b2d3a073 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:10:31.662772 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-0c850b41-67bf-4ed7-8054-ad55b2d3a073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:10:31.725372 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 215/445] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:10:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 17 19:10:31.726200 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c850b41-67bf-4ed7-8054-ad55b2d3a073 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:31.754480 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6ca8dd-e7db-483a-981d-c143835dfdb5 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:31.786276 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f7c262-8746-48fe-9ebc-669ed27a5d52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:31.787626 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f7c262-8746-48fe-9ebc-669ed27a5d52 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:31.787900 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f7c262-8746-48fe-9ebc-669ed27a5d52 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:31.788164 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f7c262-8746-48fe-9ebc-669ed27a5d52 None admin] PATH_INFO: `/v3/projects/625d29bdd4ad47ee92370dc3404fe414` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:31.789322 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f7c262-8746-48fe-9ebc-669ed27a5d52 None admin] RBAC: Authorizing `identity:get_project(project_id=625d29bdd4ad47ee92370dc3404fe414)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:31.790746 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f7c262-8746-48fe-9ebc-669ed27a5d52 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:31.792428 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 231/446] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 17 19:10:31 2022] GET /identity/v3/projects/625d29bdd4ad47ee92370dc3404fe414 => generated 302 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:31.793172 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f7c262-8746-48fe-9ebc-669ed27a5d52 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:31.822117 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c850b41-67bf-4ed7-8054-ad55b2d3a073 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:31.854718 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6649e9-94c3-4b51-95a1-7679e8eab789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:31.855379 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6649e9-94c3-4b51-95a1-7679e8eab789 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:31.855705 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6649e9-94c3-4b51-95a1-7679e8eab789 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:31.855979 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6649e9-94c3-4b51-95a1-7679e8eab789 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:31.856796 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6649e9-94c3-4b51-95a1-7679e8eab789 None admin] RBAC: Adding query filter params (group.id=7dae670cfaf346a2982f211f2796a7c7, role.id=114e6a16e63447d1924707ace5aace25, scope.project.id=625d29bdd4ad47ee92370dc3404fe414) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:31.857099 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6649e9-94c3-4b51-95a1-7679e8eab789 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:31.858533 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6649e9-94c3-4b51-95a1-7679e8eab789 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:31.864234 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 216/447] 10.222.0.58 () {56 vars in 1376 bytes} [Tue May 17 19:10:31 2022] GET /identity/v3/role_assignments?group.id=7dae670cfaf346a2982f211f2796a7c7&scope.project.id=625d29bdd4ad47ee92370dc3404fe414&role.id=114e6a16e63447d1924707ace5aace25 => generated 611 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:31.865100 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6649e9-94c3-4b51-95a1-7679e8eab789 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:31.889975 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f7c262-8746-48fe-9ebc-669ed27a5d52 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:31.928745 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daab44f8-688c-41ec-876a-c291454ceeb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:31.929552 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daab44f8-688c-41ec-876a-c291454ceeb8 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:31.929841 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daab44f8-688c-41ec-876a-c291454ceeb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:31.930097 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daab44f8-688c-41ec-876a-c291454ceeb8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:31.941352 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daab44f8-688c-41ec-876a-c291454ceeb8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 17 19:10:31.949231 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-daab44f8-688c-41ec-876a-c291454ceeb8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 17 19:10:31.949231 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:31.949231 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:31.949231 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:31.949231 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:31.949231 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:31.949231 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:31.949231 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:31.949231 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:31.949231 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:31.949231 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:31.949231 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:31.949231 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 17 19:10:31.949231 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) May 17 19:10:31.949231 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 17 19:10:31.949231 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 17 19:10:31.950108 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 17 19:10:31.950108 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 17 19:10:31.950108 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 17 19:10:31.950108 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 17 19:10:31.950108 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:31.950108 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:31.950108 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:31.950108 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:31.950108 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:31.950108 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:31.950108 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:31.950108 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:10:31.950108 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:31.950108 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:31.950108 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:31.950108 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:31.950904 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:31.950904 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:31.950904 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:31.950904 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:31.950904 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 17 19:10:31.950904 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 17 19:10:31.950904 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 17 19:10:31.950904 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 17 19:10:31.950904 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 17 19:10:31.950904 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 17 19:10:31.950904 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 17 19:10:31.950904 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:10:31.950904 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 232/448] 10.222.0.58 () {56 vars in 1096 bytes} [Tue May 17 19:10:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:10:31.956330 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daab44f8-688c-41ec-876a-c291454ceeb8 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:31.984233 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6649e9-94c3-4b51-95a1-7679e8eab789 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:32.018329 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c302cfc8-36fe-411b-b1fc-ba53992cd728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:32.019190 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c302cfc8-36fe-411b-b1fc-ba53992cd728 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:32.019462 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c302cfc8-36fe-411b-b1fc-ba53992cd728 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:32.019759 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c302cfc8-36fe-411b-b1fc-ba53992cd728 None admin] PATH_INFO: `/v3/groups/7dae670cfaf346a2982f211f2796a7c7` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:32.021524 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c302cfc8-36fe-411b-b1fc-ba53992cd728 None admin] RBAC: Authorizing `identity:get_group(group_id=7dae670cfaf346a2982f211f2796a7c7)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:32.023009 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c302cfc8-36fe-411b-b1fc-ba53992cd728 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:32.024748 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 217/449] 10.222.0.58 () {56 vars in 1180 bytes} [Tue May 17 19:10:31 2022] GET /identity/v3/groups/7dae670cfaf346a2982f211f2796a7c7 => generated 226 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:32.030554 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c302cfc8-36fe-411b-b1fc-ba53992cd728 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:32.064206 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daab44f8-688c-41ec-876a-c291454ceeb8 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:32.105616 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d07b1c-3f8a-49c4-b8d9-86db2ec3e20d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:32.106288 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d07b1c-3f8a-49c4-b8d9-86db2ec3e20d None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:32.107157 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d07b1c-3f8a-49c4-b8d9-86db2ec3e20d None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:32.107157 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d07b1c-3f8a-49c4-b8d9-86db2ec3e20d None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:32.107594 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d07b1c-3f8a-49c4-b8d9-86db2ec3e20d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:32.107899 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d07b1c-3f8a-49c4-b8d9-86db2ec3e20d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:32.109286 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d07b1c-3f8a-49c4-b8d9-86db2ec3e20d None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:32.119156 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 233/450] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 17 19:10:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:10:32.124777 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d07b1c-3f8a-49c4-b8d9-86db2ec3e20d None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:32.143194 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c302cfc8-36fe-411b-b1fc-ba53992cd728 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:32.181742 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061e7d59-332e-4ccb-b5e2-ca8d21e46bc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:32.182399 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061e7d59-332e-4ccb-b5e2-ca8d21e46bc0 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:32.182679 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061e7d59-332e-4ccb-b5e2-ca8d21e46bc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:32.182949 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061e7d59-332e-4ccb-b5e2-ca8d21e46bc0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:32.183816 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061e7d59-332e-4ccb-b5e2-ca8d21e46bc0 None admin] RBAC: Adding query filter params (group.id=7dae670cfaf346a2982f211f2796a7c7, role.id=1055a3235b0143e58f1090c1e154e5c1, scope.project.id=625d29bdd4ad47ee92370dc3404fe414) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:32.184119 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061e7d59-332e-4ccb-b5e2-ca8d21e46bc0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:32.185555 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061e7d59-332e-4ccb-b5e2-ca8d21e46bc0 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:32.192926 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 218/451] 10.222.0.58 () {56 vars in 1376 bytes} [Tue May 17 19:10:32 2022] GET /identity/v3/role_assignments?group.id=7dae670cfaf346a2982f211f2796a7c7&scope.project.id=625d29bdd4ad47ee92370dc3404fe414&role.id=1055a3235b0143e58f1090c1e154e5c1 => generated 611 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:10:32.239054 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d07b1c-3f8a-49c4-b8d9-86db2ec3e20d None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:32.278200 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7044f8-de3a-4ef6-8401-a0694de4f447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:32.278861 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7044f8-de3a-4ef6-8401-a0694de4f447 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:32.279144 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7044f8-de3a-4ef6-8401-a0694de4f447 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:32.279426 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7044f8-de3a-4ef6-8401-a0694de4f447 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:32.347778 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7044f8-de3a-4ef6-8401-a0694de4f447 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:32.349218 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7044f8-de3a-4ef6-8401-a0694de4f447 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:32.444542 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 234/452] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 17 19:10:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 17 19:10:32.450277 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061e7d59-332e-4ccb-b5e2-ca8d21e46bc0 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:32.548003 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061e7d59-332e-4ccb-b5e2-ca8d21e46bc0 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:32.580876 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e690a5e0-6b3c-4a0f-9210-ed32d5929f85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:32.581729 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e690a5e0-6b3c-4a0f-9210-ed32d5929f85 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:32.582008 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e690a5e0-6b3c-4a0f-9210-ed32d5929f85 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:32.582281 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e690a5e0-6b3c-4a0f-9210-ed32d5929f85 None admin] PATH_INFO: `/v3/projects/07bd98f7df3a477a8bf75754083485f0` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:32.583474 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e690a5e0-6b3c-4a0f-9210-ed32d5929f85 None admin] RBAC: Authorizing `identity:get_project(project_id=07bd98f7df3a477a8bf75754083485f0)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:32.584940 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e690a5e0-6b3c-4a0f-9210-ed32d5929f85 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:32.586612 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 219/453] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 17 19:10:32 2022] GET /identity/v3/projects/07bd98f7df3a477a8bf75754083485f0 => generated 344 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:32.592043 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7044f8-de3a-4ef6-8401-a0694de4f447 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:32.684460 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7044f8-de3a-4ef6-8401-a0694de4f447 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:32.716091 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebd341d-5527-43b7-aa1c-4658993cfba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:32.716920 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebd341d-5527-43b7-aa1c-4658993cfba9 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:32.717193 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebd341d-5527-43b7-aa1c-4658993cfba9 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:32.717459 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebd341d-5527-43b7-aa1c-4658993cfba9 None admin] PATH_INFO: `/v3/groups/87c9fe75443848f894d3e8f7b86160ca` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:32.718586 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebd341d-5527-43b7-aa1c-4658993cfba9 None admin] RBAC: Authorizing `identity:get_group(group_id=87c9fe75443848f894d3e8f7b86160ca)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:32.720036 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebd341d-5527-43b7-aa1c-4658993cfba9 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:32.721882 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 235/454] 10.222.0.58 () {56 vars in 1180 bytes} [Tue May 17 19:10:32 2022] GET /identity/v3/groups/87c9fe75443848f894d3e8f7b86160ca => generated 229 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:32.727786 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e690a5e0-6b3c-4a0f-9210-ed32d5929f85 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:32.824693 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e690a5e0-6b3c-4a0f-9210-ed32d5929f85 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:32.855796 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdf26aa-af63-4e8e-ab52-24ff7b513235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:32.856445 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdf26aa-af63-4e8e-ab52-24ff7b513235 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:32.856722 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdf26aa-af63-4e8e-ab52-24ff7b513235 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:32.856992 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdf26aa-af63-4e8e-ab52-24ff7b513235 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:32.857807 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdf26aa-af63-4e8e-ab52-24ff7b513235 None admin] RBAC: Adding query filter params (group.id=87c9fe75443848f894d3e8f7b86160ca, role.id=60af5088793e4379b88821a86552a344, scope.project.id=07bd98f7df3a477a8bf75754083485f0) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:32.858110 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdf26aa-af63-4e8e-ab52-24ff7b513235 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:32.859579 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdf26aa-af63-4e8e-ab52-24ff7b513235 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:32.865887 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 220/455] 10.222.0.58 () {56 vars in 1376 bytes} [Tue May 17 19:10:32 2022] GET /identity/v3/role_assignments?group.id=87c9fe75443848f894d3e8f7b86160ca&scope.project.id=07bd98f7df3a477a8bf75754083485f0&role.id=60af5088793e4379b88821a86552a344 => generated 611 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:35.664191 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaa534d-3613-45cc-8b55-c73db15f34b1 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:35.664713 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaa534d-3613-45cc-8b55-c73db15f34b1 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:35.664822 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaa534d-3613-45cc-8b55-c73db15f34b1 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:35.666345 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 236/456] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:10:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:10:35.672017 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bdc3f7-84bb-4acc-baf3-72e4affd1836 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:35.672448 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bdc3f7-84bb-4acc-baf3-72e4affd1836 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:35.672641 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bdc3f7-84bb-4acc-baf3-72e4affd1836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:35.675212 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e990e18-93c3-4a46-ac18-8b63fb8a89a6 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:35.675767 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e990e18-93c3-4a46-ac18-8b63fb8a89a6 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:35.676317 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e990e18-93c3-4a46-ac18-8b63fb8a89a6 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:35.678822 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 237/457] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:10:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:10:35.684134 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5c9d46-e5d8-4300-bf31-9d9d7c9f1f76 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:35.684674 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5c9d46-e5d8-4300-bf31-9d9d7c9f1f76 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:35.685148 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5c9d46-e5d8-4300-bf31-9d9d7c9f1f76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:35.693713 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-88bdc3f7-84bb-4acc-baf3-72e4affd1836 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:10:35.694888 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-88bdc3f7-84bb-4acc-baf3-72e4affd1836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:10:35.703736 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-ed5c9d46-e5d8-4300-bf31-9d9d7c9f1f76 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:10:35.705228 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-ed5c9d46-e5d8-4300-bf31-9d9d7c9f1f76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:10:35.763141 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 221/458] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:10:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 17 19:10:35.767005 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5589c0-e793-4242-ba6f-962a6ff225b2 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:35.767633 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5589c0-e793-4242-ba6f-962a6ff225b2 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:35.768063 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5589c0-e793-4242-ba6f-962a6ff225b2 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:35.769783 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 222/459] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:10:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:10:35.774993 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04280d5f-3550-4cdf-ab5d-b768f70b31f9 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:35.775432 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04280d5f-3550-4cdf-ab5d-b768f70b31f9 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:35.775862 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04280d5f-3550-4cdf-ab5d-b768f70b31f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:35.789370 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 238/460] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:10:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 17 19:10:35.796571 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5c9d46-e5d8-4300-bf31-9d9d7c9f1f76 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:35.813443 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-04280d5f-3550-4cdf-ab5d-b768f70b31f9 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:10:35.814862 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-04280d5f-3550-4cdf-ab5d-b768f70b31f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:10:35.884746 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 223/461] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:10:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 17 19:10:35.886331 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04280d5f-3550-4cdf-ab5d-b768f70b31f9 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:35.924085 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5c9d46-e5d8-4300-bf31-9d9d7c9f1f76 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:35.964543 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281bd88f-fd7c-441a-bf4b-ad5fe8ca2c78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:35.965607 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281bd88f-fd7c-441a-bf4b-ad5fe8ca2c78 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:35.966032 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281bd88f-fd7c-441a-bf4b-ad5fe8ca2c78 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:35.966446 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281bd88f-fd7c-441a-bf4b-ad5fe8ca2c78 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:35.979269 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281bd88f-fd7c-441a-bf4b-ad5fe8ca2c78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:35.980947 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281bd88f-fd7c-441a-bf4b-ad5fe8ca2c78 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:35.990295 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04280d5f-3550-4cdf-ab5d-b768f70b31f9 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:35.999278 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-281bd88f-fd7c-441a-bf4b-ad5fe8ca2c78 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:10:35.999278 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:35.999278 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:35.999278 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:35.999278 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:35.999278 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:35.999278 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:35.999278 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:35.999278 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:35.999278 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:35.999278 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:35.999278 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:35.999278 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 17 19:10:35.999278 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 17 19:10:35.999278 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 17 19:10:36.000375 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 17 19:10:36.000375 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:36.000375 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:36.000375 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:36.000375 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:36.000375 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:36.000375 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:36.000375 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:36.000375 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:10:36.000375 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:36.000375 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:36.000375 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:36.000375 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:36.000375 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:36.000375 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:36.000375 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:36.000375 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:36.001406 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 17 19:10:36.001406 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 17 19:10:36.001406 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:10:36.001406 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:10:36.008206 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 239/462] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:10:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:10:36.009277 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281bd88f-fd7c-441a-bf4b-ad5fe8ca2c78 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:36.025186 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd36385e-a6df-474b-a393-77c925a4f444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:36.026217 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd36385e-a6df-474b-a393-77c925a4f444 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:36.026633 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd36385e-a6df-474b-a393-77c925a4f444 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:36.027049 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd36385e-a6df-474b-a393-77c925a4f444 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:36.037259 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd36385e-a6df-474b-a393-77c925a4f444 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:36.038869 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd36385e-a6df-474b-a393-77c925a4f444 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:36.056082 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-fd36385e-a6df-474b-a393-77c925a4f444 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:10:36.056082 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:36.056082 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:36.056082 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:36.056082 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:36.056082 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:36.056082 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:36.056082 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:36.056082 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:36.056082 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:36.056082 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:36.056082 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:36.056082 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 17 19:10:36.056082 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 17 19:10:36.056082 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 17 19:10:36.057120 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 17 19:10:36.057120 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:36.057120 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:36.057120 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:36.057120 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:36.057120 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:36.057120 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:36.057120 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:36.057120 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:10:36.057120 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:36.057120 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:36.057120 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:36.057120 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:36.057120 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:36.057120 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:36.057120 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:36.057120 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:36.058088 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 17 19:10:36.058088 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 17 19:10:36.058088 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:10:36.058088 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:10:36.059560 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 224/463] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:10:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:10:36.060454 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd36385e-a6df-474b-a393-77c925a4f444 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:36.113025 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281bd88f-fd7c-441a-bf4b-ad5fe8ca2c78 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:36.147697 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b620a71e-c2b1-4185-b83b-bf9600e67ea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:36.148772 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b620a71e-c2b1-4185-b83b-bf9600e67ea7 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:36.149235 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b620a71e-c2b1-4185-b83b-bf9600e67ea7 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:36.149693 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b620a71e-c2b1-4185-b83b-bf9600e67ea7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:36.159419 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd36385e-a6df-474b-a393-77c925a4f444 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:36.160754 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b620a71e-c2b1-4185-b83b-bf9600e67ea7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:36.162307 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b620a71e-c2b1-4185-b83b-bf9600e67ea7 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:36.183393 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-b620a71e-c2b1-4185-b83b-bf9600e67ea7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:10:36.183393 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:36.183393 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:36.183393 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:36.183393 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:36.183393 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:36.183393 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:36.183393 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:36.183393 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:36.183393 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:36.183393 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:36.183393 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:36.183393 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 17 19:10:36.183393 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 17 19:10:36.183393 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 17 19:10:36.184507 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 17 19:10:36.184507 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:36.184507 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:36.184507 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:36.184507 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:36.184507 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:36.184507 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:36.184507 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:36.184507 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:10:36.184507 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:36.184507 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:36.184507 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:36.184507 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:36.184507 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:36.184507 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:36.184507 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:36.184507 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:36.185578 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 17 19:10:36.185578 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 17 19:10:36.185578 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:10:36.185578 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:10:36.192202 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 240/464] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:10:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:10:36.194048 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b620a71e-c2b1-4185-b83b-bf9600e67ea7 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:36.200314 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7dc3e8-e753-4262-9832-bd863110d23a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:36.201063 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7dc3e8-e753-4262-9832-bd863110d23a None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:36.201408 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7dc3e8-e753-4262-9832-bd863110d23a None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:36.201745 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7dc3e8-e753-4262-9832-bd863110d23a None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:36.202534 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7dc3e8-e753-4262-9832-bd863110d23a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:36.202903 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7dc3e8-e753-4262-9832-bd863110d23a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:36.204371 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7dc3e8-e753-4262-9832-bd863110d23a None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:36.222280 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 225/465] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 17 19:10:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:36.223345 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7dc3e8-e753-4262-9832-bd863110d23a None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:36.302879 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b620a71e-c2b1-4185-b83b-bf9600e67ea7 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:36.321543 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7dc3e8-e753-4262-9832-bd863110d23a None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:36.338329 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f875b994-0ee7-4a8e-87c2-ba08dcb43d49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:36.339304 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f875b994-0ee7-4a8e-87c2-ba08dcb43d49 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:36.339846 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f875b994-0ee7-4a8e-87c2-ba08dcb43d49 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:36.340297 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f875b994-0ee7-4a8e-87c2-ba08dcb43d49 None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:36.341210 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f875b994-0ee7-4a8e-87c2-ba08dcb43d49 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:36.341689 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f875b994-0ee7-4a8e-87c2-ba08dcb43d49 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:36.343253 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f875b994-0ee7-4a8e-87c2-ba08dcb43d49 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:36.355916 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 241/466] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 17 19:10:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:36.356154 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3e1308-313f-4df5-b90e-f783548731d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:36.356816 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3e1308-313f-4df5-b90e-f783548731d0 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:36.357095 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3e1308-313f-4df5-b90e-f783548731d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:36.357360 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3e1308-313f-4df5-b90e-f783548731d0 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:36.357711 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f875b994-0ee7-4a8e-87c2-ba08dcb43d49 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:36.358073 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3e1308-313f-4df5-b90e-f783548731d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:36.358387 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3e1308-313f-4df5-b90e-f783548731d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:36.359782 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3e1308-313f-4df5-b90e-f783548731d0 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:36.376430 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 226/467] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 17 19:10:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:36.377961 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e2f2fa-3d0a-445a-a3e5-45df22de4c86 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:36.378247 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e2f2fa-3d0a-445a-a3e5-45df22de4c86 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:36.378518 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e2f2fa-3d0a-445a-a3e5-45df22de4c86 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:36.380540 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 227/468] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:10:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:10:36.383036 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e2f2fa-3d0a-445a-a3e5-45df22de4c86 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:36.459334 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f875b994-0ee7-4a8e-87c2-ba08dcb43d49 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:36.483724 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e2f2fa-3d0a-445a-a3e5-45df22de4c86 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:36.492616 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71121b52-3765-4115-bdc3-439cfb1b3aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:36.493282 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71121b52-3765-4115-bdc3-439cfb1b3aec None admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:36.493555 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71121b52-3765-4115-bdc3-439cfb1b3aec None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:36.493822 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71121b52-3765-4115-bdc3-439cfb1b3aec None admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:36.494519 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71121b52-3765-4115-bdc3-439cfb1b3aec None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:36.495983 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71121b52-3765-4115-bdc3-439cfb1b3aec None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:36.516938 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719e9ca7-c8c6-4803-a86f-c2959f9b2332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:36.527715 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719e9ca7-c8c6-4803-a86f-c2959f9b2332 None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:36.527715 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719e9ca7-c8c6-4803-a86f-c2959f9b2332 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:36.527715 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719e9ca7-c8c6-4803-a86f-c2959f9b2332 None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:36.527715 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719e9ca7-c8c6-4803-a86f-c2959f9b2332 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:36.527715 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719e9ca7-c8c6-4803-a86f-c2959f9b2332 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:36.632425 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 242/469] 10.222.0.58 () {60 vars in 1132 bytes} [Tue May 17 19:10:36 2022] POST /identity/v3/users => generated 246 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:10:36.632425 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71121b52-3765-4115-bdc3-439cfb1b3aec None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:36.652680 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 228/470] 10.222.0.58 () {60 vars in 1132 bytes} [Tue May 17 19:10:36 2022] POST /identity/v3/users => generated 248 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:10:36.654643 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f84446-5787-417a-b253-82b29e926332 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:36.655265 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f84446-5787-417a-b253-82b29e926332 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:36.655572 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f84446-5787-417a-b253-82b29e926332 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:36.688876 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-51f84446-5787-417a-b253-82b29e926332 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:10:36.690120 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-51f84446-5787-417a-b253-82b29e926332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:10:36.765938 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71121b52-3765-4115-bdc3-439cfb1b3aec None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:36.808974 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 229/471] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:10:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 17 19:10:36.813198 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445ef278-2d2b-4275-9880-7ee3683b4cb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:36.813919 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445ef278-2d2b-4275-9880-7ee3683b4cb9 None admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:36.814203 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445ef278-2d2b-4275-9880-7ee3683b4cb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:36.814477 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445ef278-2d2b-4275-9880-7ee3683b4cb9 None admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:36.815222 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445ef278-2d2b-4275-9880-7ee3683b4cb9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:36.816774 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445ef278-2d2b-4275-9880-7ee3683b4cb9 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:36.850203 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f84446-5787-417a-b253-82b29e926332 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:36.903786 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 243/472] 10.222.0.58 () {60 vars in 1132 bytes} [Tue May 17 19:10:36 2022] POST /identity/v3/users => generated 248 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:10:36.972081 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f84446-5787-417a-b253-82b29e926332 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:37.011046 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a89166-8d36-4e26-b0a9-fdf82778d284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:37.011896 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a89166-8d36-4e26-b0a9-fdf82778d284 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:37.012175 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a89166-8d36-4e26-b0a9-fdf82778d284 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:37.012447 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a89166-8d36-4e26-b0a9-fdf82778d284 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:37.020278 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0a5e0c-5f49-4867-81c6-2fd606789170 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:37.020487 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0a5e0c-5f49-4867-81c6-2fd606789170 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:37.021310 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0a5e0c-5f49-4867-81c6-2fd606789170 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:37.022923 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 244/473] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:10:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:10:37.024298 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a89166-8d36-4e26-b0a9-fdf82778d284 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:37.025788 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a89166-8d36-4e26-b0a9-fdf82778d284 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:37.028525 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76e0a8b-02f0-4a19-ad15-db8008db52d9 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:37.028807 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76e0a8b-02f0-4a19-ad15-db8008db52d9 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:37.029308 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76e0a8b-02f0-4a19-ad15-db8008db52d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:37.047052 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-a1a89166-8d36-4e26-b0a9-fdf82778d284 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:10:37.047052 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:37.047052 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:37.047052 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:37.047052 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:37.047052 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:37.047052 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:37.047052 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:37.047052 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:37.047052 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:37.047052 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:37.047052 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:37.047052 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 17 19:10:37.047052 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 17 19:10:37.047052 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 17 19:10:37.048096 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 17 19:10:37.048096 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:37.048096 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:37.048096 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:37.048096 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:37.048096 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:37.048096 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:37.048096 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:37.048096 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:10:37.048096 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:37.048096 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:37.048096 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:37.048096 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:37.048096 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:37.048096 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:37.048096 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:37.048096 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:37.048922 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 17 19:10:37.048922 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 17 19:10:37.048922 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:10:37.048922 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:10:37.049143 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 230/474] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:10:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:10:37.052562 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-a76e0a8b-02f0-4a19-ad15-db8008db52d9 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:10:37.053873 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-a76e0a8b-02f0-4a19-ad15-db8008db52d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:10:37.054869 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a89166-8d36-4e26-b0a9-fdf82778d284 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:37.154517 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 245/475] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:10:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 17 19:10:37.178637 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76e0a8b-02f0-4a19-ad15-db8008db52d9 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:37.187966 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a89166-8d36-4e26-b0a9-fdf82778d284 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:37.233329 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff9b71b-d120-41a9-8296-0e92d58ce61e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:37.234003 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff9b71b-d120-41a9-8296-0e92d58ce61e None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:37.234285 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff9b71b-d120-41a9-8296-0e92d58ce61e None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:37.234559 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff9b71b-d120-41a9-8296-0e92d58ce61e None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:37.235285 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff9b71b-d120-41a9-8296-0e92d58ce61e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:37.235620 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff9b71b-d120-41a9-8296-0e92d58ce61e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:37.237012 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff9b71b-d120-41a9-8296-0e92d58ce61e None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:37.249257 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 231/476] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 17 19:10:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:37.255109 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff9b71b-d120-41a9-8296-0e92d58ce61e None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:37.326399 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76e0a8b-02f0-4a19-ad15-db8008db52d9 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:37.377233 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff9b71b-d120-41a9-8296-0e92d58ce61e None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:37.385164 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d5f471-fd45-4efe-9a77-98e15abe9d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:37.386132 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d5f471-fd45-4efe-9a77-98e15abe9d14 None admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:37.386620 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d5f471-fd45-4efe-9a77-98e15abe9d14 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:37.387087 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d5f471-fd45-4efe-9a77-98e15abe9d14 None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:37.388056 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d5f471-fd45-4efe-9a77-98e15abe9d14 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:37.389652 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d5f471-fd45-4efe-9a77-98e15abe9d14 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:37.412839 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0918ccea-f4cd-4c62-8e46-54d3b3372b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:37.413517 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0918ccea-f4cd-4c62-8e46-54d3b3372b5e None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:37.413799 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0918ccea-f4cd-4c62-8e46-54d3b3372b5e None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:37.414074 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0918ccea-f4cd-4c62-8e46-54d3b3372b5e None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:37.414839 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0918ccea-f4cd-4c62-8e46-54d3b3372b5e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:37.416351 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0918ccea-f4cd-4c62-8e46-54d3b3372b5e None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:37.436121 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.sql.core [None req-b6d5f471-fd45-4efe-9a77-98e15abe9d14 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 17 19:10:37.436121 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 17 19:10:37.436121 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [parameters: {'id': '9e8d0d3ab173475abb7f771267a0e9fd', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 17 19:10:37.436121 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80161) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 17 19:10:37.453007 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-b6d5f471-fd45-4efe-9a77-98e15abe9d14 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 17 19:10:37.453007 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:37.453007 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 17 19:10:37.453007 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self.dialect.do_execute( May 17 19:10:37.453007 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 17 19:10:37.453007 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 17 19:10:37.453007 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 17 19:10:37.453007 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application result = self._query(query) May 17 19:10:37.453007 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 17 19:10:37.453007 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application conn.query(q) May 17 19:10:37.453007 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 17 19:10:37.453007 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 17 19:10:37.453007 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 17 19:10:37.453007 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application result.read() May 17 19:10:37.453007 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 17 19:10:37.454032 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 17 19:10:37.454032 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 17 19:10:37.454032 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application packet.raise_for_error() May 17 19:10:37.454032 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 17 19:10:37.454032 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 17 19:10:37.454032 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 17 19:10:37.454032 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 17 19:10:37.454032 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 17 19:10:37.454032 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:10:37.454032 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 17 19:10:37.454032 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:10:37.454032 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:37.454032 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 17 19:10:37.454032 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 17 19:10:37.454032 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role May 17 19:10:37.454032 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return ref.to_dict() May 17 19:10:37.454032 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 17 19:10:37.454032 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application next(self.gen) May 17 19:10:37.455017 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 17 19:10:37.455017 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application yield resource May 17 19:10:37.455017 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 17 19:10:37.455017 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application next(self.gen) May 17 19:10:37.455017 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 17 19:10:37.455017 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self.session.rollback() May 17 19:10:37.455017 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 17 19:10:37.455017 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self.force_reraise() May 17 19:10:37.455017 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 17 19:10:37.455017 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise self.value May 17 19:10:37.455017 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 17 19:10:37.455017 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 17 19:10:37.455017 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 17 19:10:37.455017 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application session.commit() May 17 19:10:37.455017 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 17 19:10:37.455017 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 17 19:10:37.455017 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 17 19:10:37.455919 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self._prepare_impl() May 17 19:10:37.455919 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 17 19:10:37.455919 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self.session.flush() May 17 19:10:37.455919 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 17 19:10:37.455919 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self._flush(objects) May 17 19:10:37.455919 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 17 19:10:37.455919 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 17 19:10:37.455919 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 17 19:10:37.455919 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application compat.raise_( May 17 19:10:37.455919 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 17 19:10:37.455919 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception May 17 19:10:37.455919 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 17 19:10:37.455919 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application flush_context.execute() May 17 19:10:37.455919 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 17 19:10:37.455919 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rec.execute(self) May 17 19:10:37.455919 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 17 19:10:37.455919 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 17 19:10:37.456795 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 17 19:10:37.456795 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application _emit_insert_statements( May 17 19:10:37.456795 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 17 19:10:37.456795 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application c = connection._execute_20( May 17 19:10:37.456795 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 17 19:10:37.456795 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 17 19:10:37.456795 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 17 19:10:37.456795 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 17 19:10:37.456795 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 17 19:10:37.456795 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ret = self._execute_context( May 17 19:10:37.456795 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 17 19:10:37.456795 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 17 19:10:37.456795 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 17 19:10:37.456795 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 17 19:10:37.456795 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 17 19:10:37.456795 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception May 17 19:10:37.457591 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 17 19:10:37.457591 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self.dialect.do_execute( May 17 19:10:37.457591 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 17 19:10:37.457591 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 17 19:10:37.457591 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 17 19:10:37.457591 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application result = self._query(query) May 17 19:10:37.457591 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 17 19:10:37.457591 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application conn.query(q) May 17 19:10:37.457591 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 17 19:10:37.457591 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 17 19:10:37.457591 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 17 19:10:37.457591 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application result.read() May 17 19:10:37.457591 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 17 19:10:37.457591 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 17 19:10:37.457591 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 17 19:10:37.457591 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application packet.raise_for_error() May 17 19:10:37.458953 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 17 19:10:37.458953 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 17 19:10:37.458953 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 17 19:10:37.458953 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 17 19:10:37.458953 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 17 19:10:37.458953 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 17 19:10:37.458953 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [parameters: {'id': '9e8d0d3ab173475abb7f771267a0e9fd', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 17 19:10:37.458953 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 17 19:10:37.458953 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:10:37.458953 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 17 19:10:37.458953 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:10:37.458953 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:37.458953 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:37.458953 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:37.458953 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:37.458953 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:37.459842 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:37.459842 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:37.459842 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:37.459842 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:37.459842 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:37.459842 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:37.459842 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post May 17 19:10:37.459842 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( May 17 19:10:37.459842 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:37.459842 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:37.459842 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role May 17 19:10:37.459842 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) May 17 19:10:37.459842 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 17 19:10:37.459842 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 17 19:10:37.459842 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 17 19:10:37.459842 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:10:37.462417 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 246/477] 10.222.0.58 () {60 vars in 1131 bytes} [Tue May 17 19:10:37 2022] POST /identity/v3/roles => generated 144 bytes in 284 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) May 17 19:10:37.468421 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d5f471-fd45-4efe-9a77-98e15abe9d14 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:37.506767 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 232/478] 10.222.0.58 () {60 vars in 1132 bytes} [Tue May 17 19:10:37 2022] POST /identity/v3/users => generated 249 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:10:37.592640 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d5f471-fd45-4efe-9a77-98e15abe9d14 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:37.630555 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a89b98-6f7d-41a7-997a-41eb0a5274ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:37.631703 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a89b98-6f7d-41a7-997a-41eb0a5274ee None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:37.632234 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a89b98-6f7d-41a7-997a-41eb0a5274ee None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:37.632686 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a89b98-6f7d-41a7-997a-41eb0a5274ee None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:37.644722 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a89b98-6f7d-41a7-997a-41eb0a5274ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 17 19:10:37.654303 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-93a89b98-6f7d-41a7-997a-41eb0a5274ee None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 17 19:10:37.654303 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:37.654303 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:37.654303 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:37.654303 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:37.654303 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:37.654303 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:37.654303 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:37.654303 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:37.654303 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:37.654303 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:37.654303 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:37.654303 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 17 19:10:37.654303 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) May 17 19:10:37.654303 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 17 19:10:37.654303 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 17 19:10:37.661989 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 17 19:10:37.661989 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 17 19:10:37.661989 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 17 19:10:37.661989 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 17 19:10:37.661989 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:37.661989 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:37.661989 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:37.661989 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:37.661989 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:37.661989 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:37.661989 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:37.661989 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:10:37.661989 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:37.661989 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:37.661989 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:37.661989 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:37.662803 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:37.662803 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:37.662803 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:37.662803 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:37.662803 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 17 19:10:37.662803 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 17 19:10:37.662803 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 17 19:10:37.662803 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 17 19:10:37.662803 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 17 19:10:37.662803 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 17 19:10:37.662803 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 17 19:10:37.662803 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:10:37.662803 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 247/479] 10.222.0.58 () {56 vars in 1114 bytes} [Tue May 17 19:10:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:10:37.664648 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0918ccea-f4cd-4c62-8e46-54d3b3372b5e None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:37.774736 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0918ccea-f4cd-4c62-8e46-54d3b3372b5e None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:37.809947 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71ced92-93da-4f1c-986f-6c9ff54a7250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:37.810856 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71ced92-93da-4f1c-986f-6c9ff54a7250 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:37.811309 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71ced92-93da-4f1c-986f-6c9ff54a7250 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:37.811757 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71ced92-93da-4f1c-986f-6c9ff54a7250 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:37.812657 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71ced92-93da-4f1c-986f-6c9ff54a7250 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:37.813178 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71ced92-93da-4f1c-986f-6c9ff54a7250 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:37.814742 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71ced92-93da-4f1c-986f-6c9ff54a7250 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:37.827576 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 233/480] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 17 19:10:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:39.462000 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ad23cb-c834-4fb6-9c20-ee3ece01ab61 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:39.462844 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ad23cb-c834-4fb6-9c20-ee3ece01ab61 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:39.463257 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ad23cb-c834-4fb6-9c20-ee3ece01ab61 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:39.465000 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 248/481] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:10:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:10:39.470539 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15207dcd-e5a8-4be1-a19a-9da364fb95f3 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:39.471286 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15207dcd-e5a8-4be1-a19a-9da364fb95f3 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:39.471974 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15207dcd-e5a8-4be1-a19a-9da364fb95f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:39.489012 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-15207dcd-e5a8-4be1-a19a-9da364fb95f3 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:10:39.490900 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-15207dcd-e5a8-4be1-a19a-9da364fb95f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:10:39.557205 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 234/482] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:10:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 17 19:10:39.580338 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ad23cb-c834-4fb6-9c20-ee3ece01ab61 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:39.644794 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c98d9de-9e9d-47d6-8571-14004614cc03 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:39.645727 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c98d9de-9e9d-47d6-8571-14004614cc03 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:39.646204 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c98d9de-9e9d-47d6-8571-14004614cc03 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:39.648234 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 235/483] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:10:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:10:39.659434 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982cae1e-319f-4f07-8c00-3cd8e17fec16 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:39.659821 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982cae1e-319f-4f07-8c00-3cd8e17fec16 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:39.660094 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982cae1e-319f-4f07-8c00-3cd8e17fec16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:39.689723 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-982cae1e-319f-4f07-8c00-3cd8e17fec16 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:10:39.691500 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-982cae1e-319f-4f07-8c00-3cd8e17fec16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:10:39.691959 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ad23cb-c834-4fb6-9c20-ee3ece01ab61 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:39.728673 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baba799c-d53a-4638-b997-c73004e44964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:39.729688 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baba799c-d53a-4638-b997-c73004e44964 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:39.730105 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baba799c-d53a-4638-b997-c73004e44964 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:39.730524 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baba799c-d53a-4638-b997-c73004e44964 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:39.741787 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baba799c-d53a-4638-b997-c73004e44964 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 17 19:10:39.749583 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-baba799c-d53a-4638-b997-c73004e44964 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 17 19:10:39.749583 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:39.749583 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:39.749583 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:39.749583 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:39.749583 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:39.749583 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:39.749583 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:39.749583 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:39.749583 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:39.749583 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:39.749583 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:39.749583 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 17 19:10:39.749583 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) May 17 19:10:39.749583 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 17 19:10:39.749583 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 17 19:10:39.750715 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 17 19:10:39.750715 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 17 19:10:39.750715 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 17 19:10:39.750715 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 17 19:10:39.750715 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:39.750715 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:39.750715 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:39.750715 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:39.750715 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:39.750715 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:39.750715 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:39.750715 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:10:39.750715 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:39.750715 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:39.750715 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:39.750715 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:39.751736 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:39.751736 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:39.751736 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:39.751736 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:39.751736 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 17 19:10:39.751736 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 17 19:10:39.751736 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 17 19:10:39.751736 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 17 19:10:39.751736 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 17 19:10:39.751736 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 17 19:10:39.751736 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 17 19:10:39.751736 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:10:39.753957 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 249/484] 10.222.0.58 () {56 vars in 1102 bytes} [Tue May 17 19:10:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:10:39.759363 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baba799c-d53a-4638-b997-c73004e44964 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:39.812270 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 236/485] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:10:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 17 19:10:39.849288 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982cae1e-319f-4f07-8c00-3cd8e17fec16 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:39.948325 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baba799c-d53a-4638-b997-c73004e44964 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:39.981511 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4613ed-514b-4657-8bf2-20c2bbcde3d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:39.982406 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4613ed-514b-4657-8bf2-20c2bbcde3d2 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:39.982859 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4613ed-514b-4657-8bf2-20c2bbcde3d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:39.983295 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4613ed-514b-4657-8bf2-20c2bbcde3d2 None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:39.984221 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4613ed-514b-4657-8bf2-20c2bbcde3d2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:39.985995 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4613ed-514b-4657-8bf2-20c2bbcde3d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:39.986293 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982cae1e-319f-4f07-8c00-3cd8e17fec16 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:39.987784 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4613ed-514b-4657-8bf2-20c2bbcde3d2 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:40.000961 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 250/486] 10.222.0.58 () {56 vars in 1103 bytes} [Tue May 17 19:10:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:40.002554 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097a63d1-98fa-40a4-8b33-77660fd9b95d None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:40.003037 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097a63d1-98fa-40a4-8b33-77660fd9b95d None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:40.003480 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097a63d1-98fa-40a4-8b33-77660fd9b95d None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:40.005509 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 251/487] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:10:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:10:40.006370 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097a63d1-98fa-40a4-8b33-77660fd9b95d None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:40.030671 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f89aa74-5529-4f39-bb5a-0fb09b53faf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:40.031763 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f89aa74-5529-4f39-bb5a-0fb09b53faf5 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:40.032108 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f89aa74-5529-4f39-bb5a-0fb09b53faf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:40.032449 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f89aa74-5529-4f39-bb5a-0fb09b53faf5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:40.046955 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f89aa74-5529-4f39-bb5a-0fb09b53faf5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 17 19:10:40.057339 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-2f89aa74-5529-4f39-bb5a-0fb09b53faf5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 17 19:10:40.057339 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:40.057339 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:40.057339 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:40.057339 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:40.057339 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:40.057339 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:40.057339 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:40.057339 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:40.057339 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:40.057339 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:40.057339 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:40.057339 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 17 19:10:40.057339 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) May 17 19:10:40.057339 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 17 19:10:40.057339 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 17 19:10:40.058372 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 17 19:10:40.058372 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 17 19:10:40.058372 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 17 19:10:40.058372 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 17 19:10:40.058372 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:40.058372 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:40.058372 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:40.058372 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:40.058372 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:40.058372 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:40.058372 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:40.058372 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:10:40.058372 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:40.058372 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:40.058372 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:40.058372 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:40.059155 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:40.059155 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:40.059155 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:40.059155 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:40.059155 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 17 19:10:40.059155 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 17 19:10:40.059155 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 17 19:10:40.059155 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 17 19:10:40.059155 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 17 19:10:40.059155 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 17 19:10:40.059155 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 17 19:10:40.059155 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:10:40.063752 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 237/488] 10.222.0.58 () {56 vars in 1102 bytes} [Tue May 17 19:10:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:10:40.065039 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a606e8-57f1-4ab2-a4c3-fdfb36de134c None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:40.065455 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a606e8-57f1-4ab2-a4c3-fdfb36de134c None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:40.065733 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a606e8-57f1-4ab2-a4c3-fdfb36de134c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:40.087146 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-40a606e8-57f1-4ab2-a4c3-fdfb36de134c None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:10:40.088361 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-40a606e8-57f1-4ab2-a4c3-fdfb36de134c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:10:40.131582 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097a63d1-98fa-40a4-8b33-77660fd9b95d None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:40.158482 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 238/489] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:10:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 17 19:10:40.159661 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a606e8-57f1-4ab2-a4c3-fdfb36de134c None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:40.174556 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244b32cb-6f84-4f6e-b742-575735633b6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:40.175722 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244b32cb-6f84-4f6e-b742-575735633b6e None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:40.176191 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244b32cb-6f84-4f6e-b742-575735633b6e None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:40.176597 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244b32cb-6f84-4f6e-b742-575735633b6e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:40.188195 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244b32cb-6f84-4f6e-b742-575735633b6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:40.189936 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244b32cb-6f84-4f6e-b742-575735633b6e None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:40.208754 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-244b32cb-6f84-4f6e-b742-575735633b6e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:10:40.208754 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:40.208754 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:40.208754 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:40.208754 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:40.208754 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:40.208754 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:40.208754 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:40.208754 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:40.208754 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:40.208754 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:40.208754 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:40.208754 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 17 19:10:40.208754 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 17 19:10:40.208754 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 17 19:10:40.209832 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 17 19:10:40.209832 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:40.209832 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:40.209832 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:40.209832 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:40.209832 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:40.209832 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:40.209832 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:40.209832 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:10:40.209832 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:40.209832 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:40.209832 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:40.209832 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:40.209832 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:40.209832 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:40.209832 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:40.209832 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:40.210765 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 17 19:10:40.210765 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 17 19:10:40.210765 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:10:40.210765 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:10:40.212441 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 252/490] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:10:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:10:40.217659 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244b32cb-6f84-4f6e-b742-575735633b6e None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:40.272830 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a606e8-57f1-4ab2-a4c3-fdfb36de134c None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:40.306822 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4badb3-1ce6-4498-ada7-4aa8fc9b9ee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:40.307512 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4badb3-1ce6-4498-ada7-4aa8fc9b9ee7 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:40.307786 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4badb3-1ce6-4498-ada7-4aa8fc9b9ee7 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:40.308139 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4badb3-1ce6-4498-ada7-4aa8fc9b9ee7 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:40.308855 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4badb3-1ce6-4498-ada7-4aa8fc9b9ee7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:40.309153 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4badb3-1ce6-4498-ada7-4aa8fc9b9ee7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:40.310704 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4badb3-1ce6-4498-ada7-4aa8fc9b9ee7 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:40.319022 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 239/491] 10.222.0.58 () {56 vars in 1103 bytes} [Tue May 17 19:10:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:40.319996 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244b32cb-6f84-4f6e-b742-575735633b6e None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:40.323930 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4badb3-1ce6-4498-ada7-4aa8fc9b9ee7 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:40.355253 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b567da03-740c-4405-b683-f7e4d7d5035f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:40.356044 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b567da03-740c-4405-b683-f7e4d7d5035f None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:40.356321 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b567da03-740c-4405-b683-f7e4d7d5035f None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:40.356590 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b567da03-740c-4405-b683-f7e4d7d5035f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:40.367761 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b567da03-740c-4405-b683-f7e4d7d5035f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 17 19:10:40.376651 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-b567da03-740c-4405-b683-f7e4d7d5035f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 17 19:10:40.376651 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:40.376651 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:40.376651 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:40.376651 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:40.376651 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:40.376651 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:40.376651 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:40.376651 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:40.376651 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:40.376651 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:40.376651 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:40.376651 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 17 19:10:40.376651 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) May 17 19:10:40.376651 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 17 19:10:40.376651 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 17 19:10:40.377632 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 17 19:10:40.377632 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 17 19:10:40.377632 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 17 19:10:40.377632 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 17 19:10:40.377632 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:40.377632 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:40.377632 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:40.377632 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:40.377632 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:40.377632 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:40.377632 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:40.377632 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:10:40.377632 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:40.377632 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:40.377632 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:40.377632 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:40.378430 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:40.378430 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:40.378430 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:40.378430 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:40.378430 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 17 19:10:40.378430 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 17 19:10:40.378430 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 17 19:10:40.378430 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 17 19:10:40.378430 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 17 19:10:40.378430 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 17 19:10:40.378430 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 17 19:10:40.378430 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:10:40.378430 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 253/492] 10.222.0.58 () {56 vars in 1102 bytes} [Tue May 17 19:10:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:10:40.379084 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b567da03-740c-4405-b683-f7e4d7d5035f None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:40.429089 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4badb3-1ce6-4498-ada7-4aa8fc9b9ee7 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:40.462438 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3348b23-cd58-4cec-bed0-b1f57d30e98e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:40.463093 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3348b23-cd58-4cec-bed0-b1f57d30e98e None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:40.463368 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3348b23-cd58-4cec-bed0-b1f57d30e98e None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:40.463666 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3348b23-cd58-4cec-bed0-b1f57d30e98e None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:40.464374 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3348b23-cd58-4cec-bed0-b1f57d30e98e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:40.464672 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3348b23-cd58-4cec-bed0-b1f57d30e98e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:40.466040 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3348b23-cd58-4cec-bed0-b1f57d30e98e None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:40.477962 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 240/493] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 17 19:10:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:40.479258 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a442e200-531b-41b7-8c6e-69a3985bb283 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:40.479567 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a442e200-531b-41b7-8c6e-69a3985bb283 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:40.479838 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a442e200-531b-41b7-8c6e-69a3985bb283 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:40.481034 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 241/494] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:10:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:10:40.481705 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a442e200-531b-41b7-8c6e-69a3985bb283 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:40.492391 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b567da03-740c-4405-b683-f7e4d7d5035f None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:40.527468 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb3fdb7-59e4-467d-83ff-2677c8d7391e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:40.528191 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb3fdb7-59e4-467d-83ff-2677c8d7391e None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:40.528468 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb3fdb7-59e4-467d-83ff-2677c8d7391e None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:40.528739 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb3fdb7-59e4-467d-83ff-2677c8d7391e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:40.539000 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb3fdb7-59e4-467d-83ff-2677c8d7391e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:40.540540 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb3fdb7-59e4-467d-83ff-2677c8d7391e None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:40.556909 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-ffb3fdb7-59e4-467d-83ff-2677c8d7391e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:10:40.556909 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:40.556909 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:40.556909 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:40.556909 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:40.556909 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:40.556909 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:40.556909 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:40.556909 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:40.556909 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:40.556909 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:40.556909 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:40.556909 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 17 19:10:40.556909 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 17 19:10:40.556909 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 17 19:10:40.557792 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 17 19:10:40.557792 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:40.557792 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:40.557792 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:40.557792 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:40.557792 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:40.557792 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:40.557792 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:40.557792 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:10:40.557792 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:40.557792 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:40.557792 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:40.557792 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:40.557792 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:40.557792 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:40.557792 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:40.557792 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:40.558637 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 17 19:10:40.558637 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 17 19:10:40.558637 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:10:40.558637 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:10:40.558637 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 254/495] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:10:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:10:40.558907 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb3fdb7-59e4-467d-83ff-2677c8d7391e None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:40.586396 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a442e200-531b-41b7-8c6e-69a3985bb283 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:40.618128 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39a06bb-e3dd-4e0a-aa3f-e6c9a2728dc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:40.618779 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39a06bb-e3dd-4e0a-aa3f-e6c9a2728dc8 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:40.619057 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39a06bb-e3dd-4e0a-aa3f-e6c9a2728dc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:40.619326 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39a06bb-e3dd-4e0a-aa3f-e6c9a2728dc8 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:40.620095 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39a06bb-e3dd-4e0a-aa3f-e6c9a2728dc8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:40.620400 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39a06bb-e3dd-4e0a-aa3f-e6c9a2728dc8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:40.621764 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39a06bb-e3dd-4e0a-aa3f-e6c9a2728dc8 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:40.629424 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 242/496] 10.222.0.58 () {56 vars in 1103 bytes} [Tue May 17 19:10:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:40.630992 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdb15c9-2e1b-4ce6-82c1-6e608ab9ec19 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:40.633550 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdb15c9-2e1b-4ce6-82c1-6e608ab9ec19 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:40.633550 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdb15c9-2e1b-4ce6-82c1-6e608ab9ec19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:40.647861 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-9bdb15c9-2e1b-4ce6-82c1-6e608ab9ec19 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:10:40.649055 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-9bdb15c9-2e1b-4ce6-82c1-6e608ab9ec19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:10:40.656419 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb3fdb7-59e4-467d-83ff-2677c8d7391e None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:40.688528 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a07c83-4042-44f0-b861-ea9b055a50cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:40.689304 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a07c83-4042-44f0-b861-ea9b055a50cb None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:40.689582 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a07c83-4042-44f0-b861-ea9b055a50cb None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:40.689850 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a07c83-4042-44f0-b861-ea9b055a50cb None admin] PATH_INFO: `/v3/users/nova` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:40.697575 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a07c83-4042-44f0-b861-ea9b055a50cb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:40.699010 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a07c83-4042-44f0-b861-ea9b055a50cb None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:40.711556 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 243/497] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:10:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 17 19:10:40.712901 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944d3b00-bc80-45d0-8d30-7b5aada4c205 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:40.713185 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944d3b00-bc80-45d0-8d30-7b5aada4c205 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:40.713452 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944d3b00-bc80-45d0-8d30-7b5aada4c205 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:40.713917 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-59a07c83-4042-44f0-b861-ea9b055a50cb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 17 19:10:40.713917 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:40.713917 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:40.713917 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:40.713917 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:40.713917 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:40.713917 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:40.713917 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:40.713917 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:40.713917 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:40.713917 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:40.713917 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:40.713917 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 17 19:10:40.713917 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_user(user_id) May 17 19:10:40.713917 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 17 19:10:40.713917 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 17 19:10:40.714764 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:40.714764 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:40.714764 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 17 19:10:40.714764 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 17 19:10:40.714764 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 17 19:10:40.714764 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 17 19:10:40.714764 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:40.714764 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:40.714764 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:40.714764 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:40.714764 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:40.714764 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:10:40.714764 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:40.714764 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:40.714764 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:40.714764 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:40.714764 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:40.714764 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:40.715678 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:40.715678 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:40.715678 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 17 19:10:40.715678 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 17 19:10:40.715678 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 17 19:10:40.715678 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 17 19:10:40.715678 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 17 19:10:40.715678 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 17 19:10:40.715678 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 17 19:10:40.715678 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:10:40.716187 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 244/498] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:10:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:10:40.716187 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944d3b00-bc80-45d0-8d30-7b5aada4c205 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:40.719682 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 255/499] 10.222.0.58 () {56 vars in 1093 bytes} [Tue May 17 19:10:40 2022] GET /identity/v3/users/nova => generated 82 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:10:40.720550 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a07c83-4042-44f0-b861-ea9b055a50cb None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:40.812441 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944d3b00-bc80-45d0-8d30-7b5aada4c205 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:40.813844 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a07c83-4042-44f0-b861-ea9b055a50cb None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:40.843939 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ec57e8-0878-4532-bba3-c3c7b6210978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:40.844578 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ec57e8-0878-4532-bba3-c3c7b6210978 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:40.844851 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ec57e8-0878-4532-bba3-c3c7b6210978 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:40.845045 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a444388e-9835-4d98-a392-b43ef9a2035e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:40.845183 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ec57e8-0878-4532-bba3-c3c7b6210978 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:40.845806 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a444388e-9835-4d98-a392-b43ef9a2035e None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:40.845896 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ec57e8-0878-4532-bba3-c3c7b6210978 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:40.846086 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a444388e-9835-4d98-a392-b43ef9a2035e None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:40.846181 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ec57e8-0878-4532-bba3-c3c7b6210978 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:40.846359 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a444388e-9835-4d98-a392-b43ef9a2035e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:40.847740 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ec57e8-0878-4532-bba3-c3c7b6210978 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:40.856938 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a444388e-9835-4d98-a392-b43ef9a2035e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:40.862713 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 245/500] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 17 19:10:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:40.862889 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a444388e-9835-4d98-a392-b43ef9a2035e None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:40.862975 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c201a61-d93d-4141-8a32-ab34a1e12354 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:40.862975 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c201a61-d93d-4141-8a32-ab34a1e12354 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:40.862975 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c201a61-d93d-4141-8a32-ab34a1e12354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:40.876343 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-9c201a61-d93d-4141-8a32-ab34a1e12354 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:10:40.877549 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-9c201a61-d93d-4141-8a32-ab34a1e12354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:10:40.878407 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-a444388e-9835-4d98-a392-b43ef9a2035e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:10:40.878407 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:40.878407 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:40.878407 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:40.878407 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:40.878407 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:40.878407 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:40.878407 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:40.878407 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:40.878407 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:40.878407 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:40.878407 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:40.878407 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 17 19:10:40.878407 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 17 19:10:40.878407 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 17 19:10:40.879328 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 17 19:10:40.879328 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:40.879328 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:40.879328 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:40.879328 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:40.879328 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:40.879328 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:40.879328 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:40.879328 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:10:40.879328 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:40.879328 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:40.879328 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:40.879328 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:40.879328 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:40.879328 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:40.879328 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:40.879328 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:40.881055 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 17 19:10:40.881055 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 17 19:10:40.881055 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:10:40.881055 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:10:40.881055 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 256/501] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:10:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:10:40.882352 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a444388e-9835-4d98-a392-b43ef9a2035e None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:40.940189 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 246/502] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:10:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 17 19:10:40.941072 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c201a61-d93d-4141-8a32-ab34a1e12354 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:40.979445 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a444388e-9835-4d98-a392-b43ef9a2035e None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:41.011255 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944d004f-94d0-435d-8f92-67d68acc2b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:41.011938 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944d004f-94d0-435d-8f92-67d68acc2b2e None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:41.012219 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944d004f-94d0-435d-8f92-67d68acc2b2e None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:41.012489 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944d004f-94d0-435d-8f92-67d68acc2b2e None admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:41.013318 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944d004f-94d0-435d-8f92-67d68acc2b2e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:41.013621 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944d004f-94d0-435d-8f92-67d68acc2b2e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:41.015051 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944d004f-94d0-435d-8f92-67d68acc2b2e None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:41.032974 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c201a61-d93d-4141-8a32-ab34a1e12354 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:41.051258 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 257/503] 10.222.0.58 () {56 vars in 1133 bytes} [Tue May 17 19:10:40 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:41.053662 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944d004f-94d0-435d-8f92-67d68acc2b2e None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:41.065674 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a439f8-4e49-44f6-b5ae-41e3fd3979fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:41.066397 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a439f8-4e49-44f6-b5ae-41e3fd3979fa None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:41.066672 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a439f8-4e49-44f6-b5ae-41e3fd3979fa None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:41.066940 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a439f8-4e49-44f6-b5ae-41e3fd3979fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:41.077109 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a439f8-4e49-44f6-b5ae-41e3fd3979fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:41.078559 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a439f8-4e49-44f6-b5ae-41e3fd3979fa None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:41.095808 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-29a439f8-4e49-44f6-b5ae-41e3fd3979fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:10:41.095808 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:41.095808 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:41.095808 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:41.095808 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:41.095808 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:41.095808 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:41.095808 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:41.095808 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:41.095808 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:41.095808 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:41.095808 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:41.095808 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 17 19:10:41.095808 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 17 19:10:41.095808 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 17 19:10:41.097055 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 17 19:10:41.097055 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:41.097055 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:41.097055 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:41.097055 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:41.097055 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:41.097055 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:41.097055 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:41.097055 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:10:41.097055 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:41.097055 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:41.097055 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:41.097055 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:41.097055 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:41.097055 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:41.097055 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:41.097055 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:41.097940 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 17 19:10:41.097940 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 17 19:10:41.097940 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:10:41.097940 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:10:41.097940 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 247/504] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:10:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:10:41.097940 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a439f8-4e49-44f6-b5ae-41e3fd3979fa None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:41.152414 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944d004f-94d0-435d-8f92-67d68acc2b2e None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:41.183143 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd27b988-19b5-493f-bcd4-e721a78c2e00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:41.183942 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd27b988-19b5-493f-bcd4-e721a78c2e00 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:41.184218 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd27b988-19b5-493f-bcd4-e721a78c2e00 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:41.184484 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd27b988-19b5-493f-bcd4-e721a78c2e00 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:41.189144 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a439f8-4e49-44f6-b5ae-41e3fd3979fa None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:41.191076 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd27b988-19b5-493f-bcd4-e721a78c2e00 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:41.193065 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd27b988-19b5-493f-bcd4-e721a78c2e00 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:41.206805 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-fd27b988-19b5-493f-bcd4-e721a78c2e00 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 17 19:10:41.206805 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:41.206805 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:41.206805 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:41.206805 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:41.206805 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:41.206805 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:41.206805 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:41.206805 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:41.206805 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:41.206805 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:41.206805 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:41.206805 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 17 19:10:41.206805 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_user(user_id) May 17 19:10:41.206805 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 17 19:10:41.206805 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 17 19:10:41.207761 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:41.207761 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:41.207761 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 17 19:10:41.207761 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 17 19:10:41.207761 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 17 19:10:41.207761 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 17 19:10:41.207761 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:41.207761 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:41.207761 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:41.207761 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:41.207761 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:41.207761 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:10:41.207761 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:41.207761 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:41.207761 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:41.207761 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:41.207761 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:41.207761 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:41.209188 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:41.209188 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:41.209188 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 17 19:10:41.209188 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 17 19:10:41.209188 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 17 19:10:41.209188 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 17 19:10:41.209188 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 17 19:10:41.209188 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 17 19:10:41.209188 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 17 19:10:41.209188 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:10:41.209188 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 258/505] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 17 19:10:41 2022] GET /identity/v3/users/glance => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:10:41.209741 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd27b988-19b5-493f-bcd4-e721a78c2e00 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:41.226658 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526bce3e-eb0a-4513-b445-5c2ab78d0cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:41.227375 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526bce3e-eb0a-4513-b445-5c2ab78d0cd7 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:41.227741 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526bce3e-eb0a-4513-b445-5c2ab78d0cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:41.228013 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526bce3e-eb0a-4513-b445-5c2ab78d0cd7 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:41.228712 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526bce3e-eb0a-4513-b445-5c2ab78d0cd7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:41.229010 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526bce3e-eb0a-4513-b445-5c2ab78d0cd7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:41.230363 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526bce3e-eb0a-4513-b445-5c2ab78d0cd7 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:41.241127 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 248/506] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 17 19:10:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:41.241750 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526bce3e-eb0a-4513-b445-5c2ab78d0cd7 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:41.301551 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd27b988-19b5-493f-bcd4-e721a78c2e00 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:41.332901 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526bce3e-eb0a-4513-b445-5c2ab78d0cd7 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:41.333678 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f1de28-5bf8-4c3f-8ca3-c193c656dd3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:41.334450 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f1de28-5bf8-4c3f-8ca3-c193c656dd3d None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:41.334724 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f1de28-5bf8-4c3f-8ca3-c193c656dd3d None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:41.334989 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f1de28-5bf8-4c3f-8ca3-c193c656dd3d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:41.344582 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f1de28-5bf8-4c3f-8ca3-c193c656dd3d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 17 19:10:41.352465 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-61f1de28-5bf8-4c3f-8ca3-c193c656dd3d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 17 19:10:41.352465 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:41.352465 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:41.352465 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:41.352465 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:41.352465 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:41.352465 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:41.352465 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:41.352465 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:41.352465 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:41.352465 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:41.352465 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:41.352465 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 17 19:10:41.352465 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) May 17 19:10:41.352465 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 17 19:10:41.352465 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 17 19:10:41.353316 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 17 19:10:41.353316 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 17 19:10:41.353316 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 17 19:10:41.353316 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 17 19:10:41.353316 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:41.353316 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:41.353316 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:41.353316 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:41.353316 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:41.353316 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:41.353316 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:41.353316 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:10:41.353316 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:41.353316 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:41.353316 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:41.353316 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:41.354434 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:41.354434 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:41.354434 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:41.354434 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:41.354434 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 17 19:10:41.354434 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 17 19:10:41.354434 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 17 19:10:41.354434 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 17 19:10:41.354434 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 17 19:10:41.354434 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 17 19:10:41.354434 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 17 19:10:41.354434 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:10:41.354434 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 259/507] 10.222.0.58 () {56 vars in 1102 bytes} [Tue May 17 19:10:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:10:41.355237 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f1de28-5bf8-4c3f-8ca3-c193c656dd3d None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:41.364720 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde1f40a-83a7-4f46-bae9-5594a359d31c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:41.365354 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde1f40a-83a7-4f46-bae9-5594a359d31c None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:41.365630 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde1f40a-83a7-4f46-bae9-5594a359d31c None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:41.365902 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde1f40a-83a7-4f46-bae9-5594a359d31c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:41.428599 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde1f40a-83a7-4f46-bae9-5594a359d31c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:41.430035 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde1f40a-83a7-4f46-bae9-5594a359d31c None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:41.450562 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f1de28-5bf8-4c3f-8ca3-c193c656dd3d None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:41.481806 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbe986f-d411-4da9-b023-5e20bde6b72c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:41.482443 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbe986f-d411-4da9-b023-5e20bde6b72c None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:41.482719 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbe986f-d411-4da9-b023-5e20bde6b72c None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:41.482991 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbe986f-d411-4da9-b023-5e20bde6b72c None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:41.483731 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbe986f-d411-4da9-b023-5e20bde6b72c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:41.484035 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbe986f-d411-4da9-b023-5e20bde6b72c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:41.485381 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbe986f-d411-4da9-b023-5e20bde6b72c None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:41.496905 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 260/508] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 17 19:10:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:41.497670 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbe986f-d411-4da9-b023-5e20bde6b72c None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:41.522422 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 249/509] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 17 19:10:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 17 19:10:41.523240 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde1f40a-83a7-4f46-bae9-5594a359d31c None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:41.589781 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbe986f-d411-4da9-b023-5e20bde6b72c None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:41.613418 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde1f40a-83a7-4f46-bae9-5594a359d31c None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:41.621516 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74bfaba-8a7f-4dbc-bbc9-126149a127ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:41.622168 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74bfaba-8a7f-4dbc-bbc9-126149a127ca None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:41.622449 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74bfaba-8a7f-4dbc-bbc9-126149a127ca None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:41.622723 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74bfaba-8a7f-4dbc-bbc9-126149a127ca None admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:41.623610 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74bfaba-8a7f-4dbc-bbc9-126149a127ca None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:41.623916 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74bfaba-8a7f-4dbc-bbc9-126149a127ca None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:41.625344 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74bfaba-8a7f-4dbc-bbc9-126149a127ca None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:41.641911 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 261/510] 10.222.0.58 () {56 vars in 1137 bytes} [Tue May 17 19:10:41 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:41.642697 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74bfaba-8a7f-4dbc-bbc9-126149a127ca None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:41.647003 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d420afc-76f4-4aa8-ab9d-0944e0a8d29f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:41.647819 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d420afc-76f4-4aa8-ab9d-0944e0a8d29f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:41.648094 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d420afc-76f4-4aa8-ab9d-0944e0a8d29f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:41.648361 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d420afc-76f4-4aa8-ab9d-0944e0a8d29f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:41.655508 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d420afc-76f4-4aa8-ab9d-0944e0a8d29f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:41.656978 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d420afc-76f4-4aa8-ab9d-0944e0a8d29f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:41.672263 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-3d420afc-76f4-4aa8-ab9d-0944e0a8d29f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 17 19:10:41.672263 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:41.672263 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:41.672263 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:41.672263 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:41.672263 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:41.672263 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:41.672263 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:41.672263 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:41.672263 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:41.672263 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:41.672263 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:41.672263 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 17 19:10:41.672263 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_user(user_id) May 17 19:10:41.672263 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 17 19:10:41.672263 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 17 19:10:41.673335 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:41.673335 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:41.673335 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 17 19:10:41.673335 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 17 19:10:41.673335 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 17 19:10:41.673335 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 17 19:10:41.673335 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:41.673335 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:41.673335 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:41.673335 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:41.673335 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:41.673335 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:10:41.673335 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:41.673335 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:41.673335 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:41.673335 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:41.673335 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:41.673335 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:41.674262 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:41.674262 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:41.674262 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 17 19:10:41.674262 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 17 19:10:41.674262 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 17 19:10:41.674262 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 17 19:10:41.674262 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 17 19:10:41.674262 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 17 19:10:41.674262 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 17 19:10:41.674262 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:10:41.674262 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 250/511] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 17 19:10:41 2022] GET /identity/v3/users/cinder => generated 84 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:10:41.674262 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d420afc-76f4-4aa8-ab9d-0944e0a8d29f None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:41.739817 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74bfaba-8a7f-4dbc-bbc9-126149a127ca None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:41.767877 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d420afc-76f4-4aa8-ab9d-0944e0a8d29f None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:41.772582 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c510bca7-c125-4125-909d-dbfdd8ec524e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:41.773233 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c510bca7-c125-4125-909d-dbfdd8ec524e None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:41.773507 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c510bca7-c125-4125-909d-dbfdd8ec524e None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:41.773775 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c510bca7-c125-4125-909d-dbfdd8ec524e None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:41.774505 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c510bca7-c125-4125-909d-dbfdd8ec524e None admin] RBAC: Adding query filter params (name=service) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:41.774802 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c510bca7-c125-4125-909d-dbfdd8ec524e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:41.776190 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c510bca7-c125-4125-909d-dbfdd8ec524e None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:41.783741 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 262/512] 10.222.0.58 () {56 vars in 1103 bytes} [Tue May 17 19:10:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:41.784511 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c510bca7-c125-4125-909d-dbfdd8ec524e None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:41.811649 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89271a26-6fed-47ed-812f-8df34b1bf442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:41.812314 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89271a26-6fed-47ed-812f-8df34b1bf442 None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:41.812595 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89271a26-6fed-47ed-812f-8df34b1bf442 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:41.812868 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89271a26-6fed-47ed-812f-8df34b1bf442 None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:41.813558 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89271a26-6fed-47ed-812f-8df34b1bf442 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:41.815034 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89271a26-6fed-47ed-812f-8df34b1bf442 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:41.878645 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c510bca7-c125-4125-909d-dbfdd8ec524e None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:41.908842 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 251/513] 10.222.0.58 () {60 vars in 1132 bytes} [Tue May 17 19:10:41 2022] POST /identity/v3/users => generated 247 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:10:41.909727 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89271a26-6fed-47ed-812f-8df34b1bf442 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:41.912392 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b927ec6-46bc-4f22-9e6b-56d4e381776b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:41.913273 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b927ec6-46bc-4f22-9e6b-56d4e381776b None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:41.913629 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b927ec6-46bc-4f22-9e6b-56d4e381776b None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:41.913980 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b927ec6-46bc-4f22-9e6b-56d4e381776b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:41.925506 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b927ec6-46bc-4f22-9e6b-56d4e381776b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:41.927732 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b927ec6-46bc-4f22-9e6b-56d4e381776b None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:41.945969 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-2b927ec6-46bc-4f22-9e6b-56d4e381776b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:10:41.945969 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:41.945969 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:41.945969 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:41.945969 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:41.945969 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:41.945969 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:41.945969 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:41.945969 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:41.945969 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:41.945969 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:41.945969 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:41.945969 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 17 19:10:41.945969 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 17 19:10:41.945969 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 17 19:10:41.947142 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 17 19:10:41.947142 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:41.947142 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:41.947142 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:41.947142 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:41.947142 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:41.947142 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:41.947142 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:41.947142 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:10:41.947142 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:41.947142 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:41.947142 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:41.947142 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:41.947142 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:41.947142 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:41.947142 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:41.947142 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:41.948272 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 17 19:10:41.948272 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 17 19:10:41.948272 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:10:41.948272 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:10:41.951647 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 263/514] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:10:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:10:41.952796 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b927ec6-46bc-4f22-9e6b-56d4e381776b None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:42.021925 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89271a26-6fed-47ed-812f-8df34b1bf442 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:42.056722 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b927ec6-46bc-4f22-9e6b-56d4e381776b None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:42.057207 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d5ddee-6745-48a6-aa8b-c1502e6092bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:42.057867 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d5ddee-6745-48a6-aa8b-c1502e6092bb None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:42.058144 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d5ddee-6745-48a6-aa8b-c1502e6092bb None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:42.058415 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d5ddee-6745-48a6-aa8b-c1502e6092bb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:42.095333 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2fbfd2-21a0-4d77-ac38-d899503cc305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:42.096889 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2fbfd2-21a0-4d77-ac38-d899503cc305 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:42.096889 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2fbfd2-21a0-4d77-ac38-d899503cc305 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:42.096889 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2fbfd2-21a0-4d77-ac38-d899503cc305 None admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:42.097870 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2fbfd2-21a0-4d77-ac38-d899503cc305 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:42.097968 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2fbfd2-21a0-4d77-ac38-d899503cc305 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:42.099386 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2fbfd2-21a0-4d77-ac38-d899503cc305 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:42.117074 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 264/515] 10.222.0.58 () {56 vars in 1137 bytes} [Tue May 17 19:10:41 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:42.117919 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2fbfd2-21a0-4d77-ac38-d899503cc305 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:42.128773 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d5ddee-6745-48a6-aa8b-c1502e6092bb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:42.130709 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d5ddee-6745-48a6-aa8b-c1502e6092bb None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:42.222966 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2fbfd2-21a0-4d77-ac38-d899503cc305 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:42.233553 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 252/516] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 17 19:10:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 17 19:10:42.233553 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d5ddee-6745-48a6-aa8b-c1502e6092bb None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:42.260426 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e59ff6-f67b-46a8-ae35-ec2bd996e907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:42.261193 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e59ff6-f67b-46a8-ae35-ec2bd996e907 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:42.261469 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e59ff6-f67b-46a8-ae35-ec2bd996e907 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:42.261737 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e59ff6-f67b-46a8-ae35-ec2bd996e907 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:42.271412 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e59ff6-f67b-46a8-ae35-ec2bd996e907 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:42.272866 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e59ff6-f67b-46a8-ae35-ec2bd996e907 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:42.290046 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-d7e59ff6-f67b-46a8-ae35-ec2bd996e907 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:10:42.290046 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:42.290046 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:42.290046 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:42.290046 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:42.290046 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:42.290046 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:42.290046 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:42.290046 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:42.290046 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:42.290046 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:42.290046 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:42.290046 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 17 19:10:42.290046 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 17 19:10:42.290046 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 17 19:10:42.290899 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 17 19:10:42.290899 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:42.290899 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:42.290899 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:42.290899 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:42.290899 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:42.290899 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:42.290899 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:42.290899 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:10:42.290899 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:42.290899 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:42.290899 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:42.290899 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:42.290899 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:42.290899 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:42.290899 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:42.290899 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:42.291792 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 17 19:10:42.291792 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 17 19:10:42.291792 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:10:42.291792 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:10:42.295845 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 265/517] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:10:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:10:42.296425 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e59ff6-f67b-46a8-ae35-ec2bd996e907 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:42.327549 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d5ddee-6745-48a6-aa8b-c1502e6092bb None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:42.360638 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604d3ea5-53f0-4b44-8381-c3e433338904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:42.361280 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604d3ea5-53f0-4b44-8381-c3e433338904 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:42.361565 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604d3ea5-53f0-4b44-8381-c3e433338904 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:42.361829 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604d3ea5-53f0-4b44-8381-c3e433338904 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:42.362538 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604d3ea5-53f0-4b44-8381-c3e433338904 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:42.362838 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604d3ea5-53f0-4b44-8381-c3e433338904 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:42.364776 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604d3ea5-53f0-4b44-8381-c3e433338904 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:42.376080 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 253/518] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 17 19:10:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:42.377066 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604d3ea5-53f0-4b44-8381-c3e433338904 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:42.396718 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e59ff6-f67b-46a8-ae35-ec2bd996e907 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:42.430253 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0be53b4-3883-43d3-9933-61daaac7e0fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:42.430896 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0be53b4-3883-43d3-9933-61daaac7e0fa None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:42.431174 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0be53b4-3883-43d3-9933-61daaac7e0fa None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:42.431447 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0be53b4-3883-43d3-9933-61daaac7e0fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:42.475338 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604d3ea5-53f0-4b44-8381-c3e433338904 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:42.498829 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0be53b4-3883-43d3-9933-61daaac7e0fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:42.500339 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0be53b4-3883-43d3-9933-61daaac7e0fa None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:42.508712 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ece67d-089a-4d7f-a174-78ad14f12a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:42.509491 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ece67d-089a-4d7f-a174-78ad14f12a32 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:42.509764 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ece67d-089a-4d7f-a174-78ad14f12a32 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:42.510031 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ece67d-089a-4d7f-a174-78ad14f12a32 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:42.520365 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ece67d-089a-4d7f-a174-78ad14f12a32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:42.521878 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ece67d-089a-4d7f-a174-78ad14f12a32 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:42.541697 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-d4ece67d-089a-4d7f-a174-78ad14f12a32 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:10:42.541697 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:42.541697 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:42.541697 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:42.541697 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:42.541697 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:42.541697 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:42.541697 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:42.541697 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:42.541697 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:42.541697 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:42.541697 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:42.541697 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 17 19:10:42.541697 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 17 19:10:42.541697 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 17 19:10:42.542985 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 17 19:10:42.542985 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:42.542985 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:42.542985 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:42.542985 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:42.542985 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:42.542985 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:42.542985 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:42.542985 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:10:42.542985 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:42.542985 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:42.542985 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:42.542985 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:42.542985 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:42.542985 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:42.542985 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:42.542985 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:42.543873 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 17 19:10:42.543873 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 17 19:10:42.543873 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:10:42.543873 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:10:42.543873 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 254/519] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:10:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:10:42.544138 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ece67d-089a-4d7f-a174-78ad14f12a32 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:42.599451 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 266/520] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 17 19:10:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 17 19:10:42.600333 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0be53b4-3883-43d3-9933-61daaac7e0fa None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:42.641021 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ece67d-089a-4d7f-a174-78ad14f12a32 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:42.674777 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190fcd1d-5548-432d-aac5-10446c64bfb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:42.675467 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190fcd1d-5548-432d-aac5-10446c64bfb8 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:42.675841 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190fcd1d-5548-432d-aac5-10446c64bfb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:42.676184 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190fcd1d-5548-432d-aac5-10446c64bfb8 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:42.677058 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190fcd1d-5548-432d-aac5-10446c64bfb8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:42.677426 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190fcd1d-5548-432d-aac5-10446c64bfb8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:42.678886 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190fcd1d-5548-432d-aac5-10446c64bfb8 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:42.690558 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 255/521] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 17 19:10:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:42.691349 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190fcd1d-5548-432d-aac5-10446c64bfb8 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:42.768096 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0be53b4-3883-43d3-9933-61daaac7e0fa None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:42.804742 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870468cb-6300-4eb9-9c8d-ae896f199248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:42.805529 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870468cb-6300-4eb9-9c8d-ae896f199248 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:42.805807 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870468cb-6300-4eb9-9c8d-ae896f199248 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:42.806076 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870468cb-6300-4eb9-9c8d-ae896f199248 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:42.812576 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870468cb-6300-4eb9-9c8d-ae896f199248 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:42.814126 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870468cb-6300-4eb9-9c8d-ae896f199248 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:42.829122 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-870468cb-6300-4eb9-9c8d-ae896f199248 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 17 19:10:42.829122 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:42.829122 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:42.829122 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:42.829122 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:42.829122 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:42.829122 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:42.829122 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:42.829122 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:42.829122 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:42.829122 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:42.829122 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:42.829122 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 17 19:10:42.829122 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(project_id) May 17 19:10:42.829122 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 17 19:10:42.829972 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 17 19:10:42.829972 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:42.829972 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:42.829972 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:42.829972 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:42.829972 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:42.829972 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:42.829972 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:42.829972 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:10:42.829972 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:42.829972 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:42.829972 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:42.829972 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:42.829972 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:42.829972 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:42.829972 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:42.829972 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:42.836318 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 17 19:10:42.836318 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 17 19:10:42.836318 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 17 19:10:42.836318 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 17 19:10:42.836318 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 17 19:10:42.836318 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 17 19:10:42.836318 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 17 19:10:42.836318 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:10:42.836824 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190fcd1d-5548-432d-aac5-10446c64bfb8 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:42.836907 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 267/522] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 17 19:10:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:10:42.836907 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870468cb-6300-4eb9-9c8d-ae896f199248 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:42.864809 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d642b90f-a291-47f4-b3b5-b7cf528499e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:42.865460 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d642b90f-a291-47f4-b3b5-b7cf528499e5 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:42.865735 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d642b90f-a291-47f4-b3b5-b7cf528499e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:42.866008 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d642b90f-a291-47f4-b3b5-b7cf528499e5 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:42.866713 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d642b90f-a291-47f4-b3b5-b7cf528499e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:42.867022 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d642b90f-a291-47f4-b3b5-b7cf528499e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:42.868414 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d642b90f-a291-47f4-b3b5-b7cf528499e5 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:42.879350 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 256/523] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 17 19:10:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:42.880888 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d642b90f-a291-47f4-b3b5-b7cf528499e5 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:42.937275 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870468cb-6300-4eb9-9c8d-ae896f199248 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:42.971254 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68629cb3-3de6-494b-bd80-7416e04eb669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:42.972346 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68629cb3-3de6-494b-bd80-7416e04eb669 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:42.972600 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68629cb3-3de6-494b-bd80-7416e04eb669 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:42.972871 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68629cb3-3de6-494b-bd80-7416e04eb669 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:42.980463 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d642b90f-a291-47f4-b3b5-b7cf528499e5 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:42.982971 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68629cb3-3de6-494b-bd80-7416e04eb669 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:42.984443 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68629cb3-3de6-494b-bd80-7416e04eb669 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:43.001480 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-68629cb3-3de6-494b-bd80-7416e04eb669 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:10:43.001480 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:43.001480 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:43.001480 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:43.001480 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:43.001480 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:43.001480 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:43.001480 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:43.001480 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:43.001480 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:43.001480 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:43.001480 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:43.001480 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 17 19:10:43.001480 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 17 19:10:43.001480 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 17 19:10:43.002361 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 17 19:10:43.002361 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:43.002361 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:43.002361 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:43.002361 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:43.002361 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:43.002361 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:43.002361 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:43.002361 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:10:43.002361 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:43.002361 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:43.002361 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:43.002361 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:43.002361 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:43.002361 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:43.002361 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:43.002361 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:43.003214 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 17 19:10:43.003214 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 17 19:10:43.003214 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:10:43.003214 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:10:43.007300 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 268/524] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:10:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:10:43.008160 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68629cb3-3de6-494b-bd80-7416e04eb669 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:43.013432 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaf0bcf-d9ab-4757-82b1-16e881f971e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:43.014224 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaf0bcf-d9ab-4757-82b1-16e881f971e9 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:43.014505 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaf0bcf-d9ab-4757-82b1-16e881f971e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:43.014781 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaf0bcf-d9ab-4757-82b1-16e881f971e9 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:43.023041 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aaf0bcf-d9ab-4757-82b1-16e881f971e9 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:43.024516 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aaf0bcf-d9ab-4757-82b1-16e881f971e9 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:43.038361 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-5aaf0bcf-d9ab-4757-82b1-16e881f971e9 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 17 19:10:43.038361 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:43.038361 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:43.038361 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:43.038361 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:43.038361 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:43.038361 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:43.038361 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:43.038361 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:43.038361 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:43.038361 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:43.038361 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:43.038361 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 17 19:10:43.038361 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_user(user_id) May 17 19:10:43.038361 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 17 19:10:43.038361 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 17 19:10:43.039255 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:43.039255 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:43.039255 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 17 19:10:43.039255 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 17 19:10:43.039255 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 17 19:10:43.039255 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 17 19:10:43.039255 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:43.039255 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:43.039255 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:43.039255 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:43.039255 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:43.039255 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:10:43.039255 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:43.039255 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:43.039255 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:43.039255 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:43.039255 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:43.039255 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:43.040160 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:43.040160 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:43.040160 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 17 19:10:43.040160 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 17 19:10:43.040160 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 17 19:10:43.040160 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 17 19:10:43.040160 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 17 19:10:43.040160 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 17 19:10:43.040160 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 17 19:10:43.040160 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:10:43.040160 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 257/525] 10.222.0.58 () {56 vars in 1102 bytes} [Tue May 17 19:10:42 2022] GET /identity/v3/users/neutron => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:10:43.041205 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaf0bcf-d9ab-4757-82b1-16e881f971e9 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:43.116745 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68629cb3-3de6-494b-bd80-7416e04eb669 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:43.142143 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaf0bcf-d9ab-4757-82b1-16e881f971e9 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:43.148836 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82108af9-5b49-443a-81ea-7a610ed19676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:43.149494 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82108af9-5b49-443a-81ea-7a610ed19676 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:43.149776 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82108af9-5b49-443a-81ea-7a610ed19676 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:43.150051 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82108af9-5b49-443a-81ea-7a610ed19676 None admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:43.150838 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82108af9-5b49-443a-81ea-7a610ed19676 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:43.151174 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82108af9-5b49-443a-81ea-7a610ed19676 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:43.152668 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82108af9-5b49-443a-81ea-7a610ed19676 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:43.175252 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 269/526] 10.222.0.58 () {56 vars in 1148 bytes} [Tue May 17 19:10:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:43.175252 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82108af9-5b49-443a-81ea-7a610ed19676 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:43.175730 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443b1bff-a0bb-474b-9235-86435511078e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:43.176550 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443b1bff-a0bb-474b-9235-86435511078e None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:43.176825 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443b1bff-a0bb-474b-9235-86435511078e None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:43.177092 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443b1bff-a0bb-474b-9235-86435511078e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:43.183902 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443b1bff-a0bb-474b-9235-86435511078e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:43.185359 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443b1bff-a0bb-474b-9235-86435511078e None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:43.200615 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-443b1bff-a0bb-474b-9235-86435511078e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 17 19:10:43.200615 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:43.200615 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:43.200615 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:43.200615 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:43.200615 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:43.200615 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:43.200615 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:43.200615 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:43.200615 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:43.200615 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:43.200615 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:43.200615 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 17 19:10:43.200615 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(project_id) May 17 19:10:43.200615 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 17 19:10:43.201452 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 17 19:10:43.201452 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:43.201452 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:43.201452 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:43.201452 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:43.201452 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:43.201452 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:43.201452 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:43.201452 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:10:43.201452 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:43.201452 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:43.201452 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:43.201452 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:43.201452 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:43.201452 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:43.201452 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:43.201452 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:43.202265 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 17 19:10:43.202265 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 17 19:10:43.202265 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 17 19:10:43.202265 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 17 19:10:43.202265 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 17 19:10:43.202265 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 17 19:10:43.202265 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 17 19:10:43.202265 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:10:43.202265 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 258/527] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 17 19:10:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:10:43.202747 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443b1bff-a0bb-474b-9235-86435511078e None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:43.276849 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82108af9-5b49-443a-81ea-7a610ed19676 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:43.299573 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443b1bff-a0bb-474b-9235-86435511078e None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:43.309487 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423312d8-a62b-4fbf-add3-d748c492a3f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:43.310142 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423312d8-a62b-4fbf-add3-d748c492a3f7 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:43.310422 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423312d8-a62b-4fbf-add3-d748c492a3f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:43.310692 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423312d8-a62b-4fbf-add3-d748c492a3f7 None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:43.311432 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423312d8-a62b-4fbf-add3-d748c492a3f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:43.311752 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423312d8-a62b-4fbf-add3-d748c492a3f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:43.313153 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423312d8-a62b-4fbf-add3-d748c492a3f7 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:43.328772 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 270/528] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 17 19:10:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:43.328772 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423312d8-a62b-4fbf-add3-d748c492a3f7 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:43.332544 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ec5161-bd97-4789-a87e-1dc1083798b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:43.333248 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ec5161-bd97-4789-a87e-1dc1083798b3 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:43.333520 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ec5161-bd97-4789-a87e-1dc1083798b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:43.333810 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ec5161-bd97-4789-a87e-1dc1083798b3 None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:43.334611 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ec5161-bd97-4789-a87e-1dc1083798b3 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:43.334914 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ec5161-bd97-4789-a87e-1dc1083798b3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:43.336875 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ec5161-bd97-4789-a87e-1dc1083798b3 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:43.372172 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 259/529] 10.222.0.58 () {56 vars in 1139 bytes} [Tue May 17 19:10:43 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:43.373283 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ec5161-bd97-4789-a87e-1dc1083798b3 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:43.426719 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423312d8-a62b-4fbf-add3-d748c492a3f7 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:43.459703 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0a75df-73a4-446c-b717-ccd7cca9abc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:43.460282 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0a75df-73a4-446c-b717-ccd7cca9abc7 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:43.460564 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0a75df-73a4-446c-b717-ccd7cca9abc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:43.460840 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0a75df-73a4-446c-b717-ccd7cca9abc7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:43.461669 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0a75df-73a4-446c-b717-ccd7cca9abc7 None admin] RBAC: Adding query filter params (role.id=1754a95b778c4e20b53b20cb52ada92b, scope.project.id=1a9d3a0631284c9fb113266036993e2d, user.id=65c1f878b0dc4d63a4f5dcb12fac1dc1) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:43.461980 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0a75df-73a4-446c-b717-ccd7cca9abc7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:43.463409 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0a75df-73a4-446c-b717-ccd7cca9abc7 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:43.466682 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ec5161-bd97-4789-a87e-1dc1083798b3 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:43.468712 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 271/530] 10.222.0.58 () {56 vars in 1374 bytes} [Tue May 17 19:10:43 2022] GET /identity/v3/role_assignments?user.id=65c1f878b0dc4d63a4f5dcb12fac1dc1&scope.project.id=1a9d3a0631284c9fb113266036993e2d&role.id=1754a95b778c4e20b53b20cb52ada92b => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:43.479794 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0a75df-73a4-446c-b717-ccd7cca9abc7 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:43.516333 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8341113b-311e-4c40-91bd-fa9a903c93bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:43.517085 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8341113b-311e-4c40-91bd-fa9a903c93bf None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:43.517449 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8341113b-311e-4c40-91bd-fa9a903c93bf None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:43.517798 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8341113b-311e-4c40-91bd-fa9a903c93bf None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:43.518591 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8341113b-311e-4c40-91bd-fa9a903c93bf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:43.518895 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8341113b-311e-4c40-91bd-fa9a903c93bf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:43.521129 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8341113b-311e-4c40-91bd-fa9a903c93bf None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:43.541264 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 260/531] 10.222.0.58 () {56 vars in 1148 bytes} [Tue May 17 19:10:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:43.542151 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8341113b-311e-4c40-91bd-fa9a903c93bf None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:43.595082 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0a75df-73a4-446c-b717-ccd7cca9abc7 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:43.632159 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fdcd25-e45c-43cb-b985-f10998589e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:43.633102 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fdcd25-e45c-43cb-b985-f10998589e05 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:43.633451 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fdcd25-e45c-43cb-b985-f10998589e05 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:43.633888 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fdcd25-e45c-43cb-b985-f10998589e05 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:43.641571 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fdcd25-e45c-43cb-b985-f10998589e05 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:43.643210 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fdcd25-e45c-43cb-b985-f10998589e05 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:43.649913 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8341113b-311e-4c40-91bd-fa9a903c93bf None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:43.659786 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-54fdcd25-e45c-43cb-b985-f10998589e05 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 17 19:10:43.659786 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:43.659786 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:43.659786 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:43.659786 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:43.659786 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:43.659786 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:43.659786 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:43.659786 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:43.659786 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:43.659786 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:43.659786 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:43.659786 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 17 19:10:43.659786 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(project_id) May 17 19:10:43.659786 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 17 19:10:43.660913 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 17 19:10:43.660913 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:43.660913 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:43.660913 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:43.660913 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:43.660913 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:43.660913 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:43.660913 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:43.660913 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:10:43.660913 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:43.660913 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:43.660913 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:43.660913 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:43.660913 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:43.660913 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:43.660913 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:43.660913 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:43.661940 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 17 19:10:43.661940 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 17 19:10:43.661940 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 17 19:10:43.661940 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 17 19:10:43.661940 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 17 19:10:43.661940 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 17 19:10:43.661940 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 17 19:10:43.661940 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:10:43.667512 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 272/532] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 17 19:10:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:10:43.667512 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fdcd25-e45c-43cb-b985-f10998589e05 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:43.685750 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37db5cd-639b-4656-8be1-96aeac3ff8ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:43.686481 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37db5cd-639b-4656-8be1-96aeac3ff8ea None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:43.686762 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37db5cd-639b-4656-8be1-96aeac3ff8ea None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:43.687031 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37db5cd-639b-4656-8be1-96aeac3ff8ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:43.756533 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37db5cd-639b-4656-8be1-96aeac3ff8ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:43.757921 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37db5cd-639b-4656-8be1-96aeac3ff8ea None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:43.766552 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fdcd25-e45c-43cb-b985-f10998589e05 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:43.803338 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333e8677-6cb1-4829-b2dd-ce9870743a90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:43.804176 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333e8677-6cb1-4829-b2dd-ce9870743a90 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:43.804558 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333e8677-6cb1-4829-b2dd-ce9870743a90 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:43.804960 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333e8677-6cb1-4829-b2dd-ce9870743a90 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:43.805901 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333e8677-6cb1-4829-b2dd-ce9870743a90 None admin] RBAC: Adding query filter params (role.id=1754a95b778c4e20b53b20cb52ada92b, scope.project.id=1a9d3a0631284c9fb113266036993e2d, user.id=57e888884d2b4629bfd12d1034ce5be4) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:43.806313 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333e8677-6cb1-4829-b2dd-ce9870743a90 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:43.807894 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333e8677-6cb1-4829-b2dd-ce9870743a90 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:43.819812 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 273/533] 10.222.0.58 () {56 vars in 1374 bytes} [Tue May 17 19:10:43 2022] GET /identity/v3/role_assignments?user.id=57e888884d2b4629bfd12d1034ce5be4&scope.project.id=1a9d3a0631284c9fb113266036993e2d&role.id=1754a95b778c4e20b53b20cb52ada92b => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:43.820940 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333e8677-6cb1-4829-b2dd-ce9870743a90 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:43.864792 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 261/534] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 17 19:10:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 17 19:10:43.870516 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37db5cd-639b-4656-8be1-96aeac3ff8ea None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:43.966893 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333e8677-6cb1-4829-b2dd-ce9870743a90 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:43.977905 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37db5cd-639b-4656-8be1-96aeac3ff8ea None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:44.007775 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb2a547-4f25-4e0c-9cda-282d7457e91b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:44.008786 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb2a547-4f25-4e0c-9cda-282d7457e91b None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:44.009362 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb2a547-4f25-4e0c-9cda-282d7457e91b None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:44.009854 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb2a547-4f25-4e0c-9cda-282d7457e91b None admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:44.010854 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb2a547-4f25-4e0c-9cda-282d7457e91b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:44.011361 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb2a547-4f25-4e0c-9cda-282d7457e91b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:44.014629 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb2a547-4f25-4e0c-9cda-282d7457e91b None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:44.028616 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 274/535] 10.222.0.58 () {56 vars in 1148 bytes} [Tue May 17 19:10:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:44.037114 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb2a547-4f25-4e0c-9cda-282d7457e91b None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:44.039962 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2891524f-fff9-4f8d-8c97-7676d5123d4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:44.039962 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2891524f-fff9-4f8d-8c97-7676d5123d4b None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:44.039962 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2891524f-fff9-4f8d-8c97-7676d5123d4b None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:44.039962 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2891524f-fff9-4f8d-8c97-7676d5123d4b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:44.051092 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2891524f-fff9-4f8d-8c97-7676d5123d4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:44.052679 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2891524f-fff9-4f8d-8c97-7676d5123d4b None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:44.072605 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-2891524f-fff9-4f8d-8c97-7676d5123d4b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:10:44.072605 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:44.072605 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:44.072605 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:44.072605 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:44.072605 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:44.072605 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:44.072605 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:44.072605 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:44.072605 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:44.072605 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:44.072605 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:44.072605 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 17 19:10:44.072605 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 17 19:10:44.072605 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 17 19:10:44.073517 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 17 19:10:44.073517 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:44.073517 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:44.073517 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:44.073517 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:44.073517 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:44.073517 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:44.073517 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:44.073517 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:10:44.073517 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:44.073517 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:44.073517 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:44.073517 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:44.073517 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:44.073517 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:44.073517 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:44.073517 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:44.074393 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 17 19:10:44.074393 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 17 19:10:44.074393 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:10:44.074393 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:10:44.074393 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 262/536] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:10:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 17 19:10:44.079622 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2891524f-fff9-4f8d-8c97-7676d5123d4b None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:44.187840 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2891524f-fff9-4f8d-8c97-7676d5123d4b None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:44.219438 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb2a547-4f25-4e0c-9cda-282d7457e91b None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:44.231799 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dea6d8-f761-49c8-9736-d3b5eef7a633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:44.232672 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dea6d8-f761-49c8-9736-d3b5eef7a633 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:44.232957 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dea6d8-f761-49c8-9736-d3b5eef7a633 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:44.233603 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dea6d8-f761-49c8-9736-d3b5eef7a633 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:44.234604 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dea6d8-f761-49c8-9736-d3b5eef7a633 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:44.234914 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dea6d8-f761-49c8-9736-d3b5eef7a633 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:44.236764 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dea6d8-f761-49c8-9736-d3b5eef7a633 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:44.249191 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 263/537] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 17 19:10:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:10:44.250486 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e9cffe-9646-4227-9bde-c4bf231b8620 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:44.250767 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e9cffe-9646-4227-9bde-c4bf231b8620 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:44.251038 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e9cffe-9646-4227-9bde-c4bf231b8620 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:44.256600 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 264/538] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:10:44 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:10:44.260365 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e9cffe-9646-4227-9bde-c4bf231b8620 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:44.264363 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4dbb04-64e3-442b-8390-4602fecbd79e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:44.265180 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4dbb04-64e3-442b-8390-4602fecbd79e None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:44.265553 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4dbb04-64e3-442b-8390-4602fecbd79e None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:44.265924 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4dbb04-64e3-442b-8390-4602fecbd79e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:44.266944 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4dbb04-64e3-442b-8390-4602fecbd79e None admin] RBAC: Adding query filter params (role.id=1754a95b778c4e20b53b20cb52ada92b, scope.project.id=1a9d3a0631284c9fb113266036993e2d, user.id=d05409b38ae346458abfe41ebda1c59e) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:44.267366 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4dbb04-64e3-442b-8390-4602fecbd79e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:44.269110 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4dbb04-64e3-442b-8390-4602fecbd79e None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:44.276146 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 275/539] 10.222.0.58 () {56 vars in 1374 bytes} [Tue May 17 19:10:44 2022] GET /identity/v3/role_assignments?user.id=d05409b38ae346458abfe41ebda1c59e&scope.project.id=1a9d3a0631284c9fb113266036993e2d&role.id=1754a95b778c4e20b53b20cb52ada92b => generated 259 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:44.277646 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9c8fe9-f3af-4230-b8ee-a94b796ecd86 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:44.278034 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9c8fe9-f3af-4230-b8ee-a94b796ecd86 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:44.278449 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9c8fe9-f3af-4230-b8ee-a94b796ecd86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:44.296229 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-6f9c8fe9-f3af-4230-b8ee-a94b796ecd86 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:10:44.297634 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-6f9c8fe9-f3af-4230-b8ee-a94b796ecd86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:10:44.432024 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e9cffe-9646-4227-9bde-c4bf231b8620 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:44.439445 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 276/540] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:10:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 17 19:10:44.470040 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e3c621-f339-40e1-8b31-4b52e258af21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:44.471022 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e3c621-f339-40e1-8b31-4b52e258af21 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:44.471656 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e3c621-f339-40e1-8b31-4b52e258af21 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:44.472072 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e3c621-f339-40e1-8b31-4b52e258af21 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:44.480775 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e3c621-f339-40e1-8b31-4b52e258af21 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:44.482383 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e3c621-f339-40e1-8b31-4b52e258af21 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:44.486316 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9c8fe9-f3af-4230-b8ee-a94b796ecd86 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:44.517757 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-89e3c621-f339-40e1-8b31-4b52e258af21 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 17 19:10:44.517757 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:44.517757 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:44.517757 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:44.517757 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:44.517757 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:44.517757 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:44.517757 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:44.517757 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:44.517757 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:44.517757 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:44.517757 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:44.517757 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 17 19:10:44.517757 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(project_id) May 17 19:10:44.517757 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 17 19:10:44.519811 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 17 19:10:44.519811 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:44.519811 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:44.519811 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:44.519811 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:44.519811 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:44.519811 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:44.519811 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:44.519811 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:10:44.519811 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:44.519811 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:44.519811 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:44.519811 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:44.519811 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:44.519811 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:44.519811 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:44.519811 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:44.521280 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 17 19:10:44.521280 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 17 19:10:44.521280 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 17 19:10:44.521280 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 17 19:10:44.521280 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 17 19:10:44.521280 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 17 19:10:44.521280 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 17 19:10:44.521280 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:10:44.523674 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 265/541] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 17 19:10:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:10:44.530490 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e3c621-f339-40e1-8b31-4b52e258af21 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:44.607383 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9c8fe9-f3af-4230-b8ee-a94b796ecd86 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:44.649841 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2624860d-2087-4598-8835-792491f88e3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:44.650641 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2624860d-2087-4598-8835-792491f88e3d None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:44.650919 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2624860d-2087-4598-8835-792491f88e3d None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:44.651192 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2624860d-2087-4598-8835-792491f88e3d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:44.658674 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e3c621-f339-40e1-8b31-4b52e258af21 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:44.662015 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2624860d-2087-4598-8835-792491f88e3d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 17 19:10:44.669968 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-2624860d-2087-4598-8835-792491f88e3d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 17 19:10:44.669968 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:44.669968 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:44.669968 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:44.669968 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:44.669968 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:44.669968 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:44.669968 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:44.669968 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:44.669968 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:44.669968 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:44.669968 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:44.669968 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 17 19:10:44.669968 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) May 17 19:10:44.669968 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 17 19:10:44.669968 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 17 19:10:44.672079 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 17 19:10:44.672079 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 17 19:10:44.672079 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 17 19:10:44.672079 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 17 19:10:44.672079 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:44.672079 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:44.672079 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:44.672079 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:44.672079 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:44.672079 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:44.672079 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:44.672079 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:10:44.672079 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:44.672079 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:44.672079 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:44.672079 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:44.674018 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:44.674018 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:44.674018 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:44.674018 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:44.674018 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 17 19:10:44.674018 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 17 19:10:44.674018 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 17 19:10:44.674018 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 17 19:10:44.674018 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 17 19:10:44.674018 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 17 19:10:44.674018 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 17 19:10:44.674018 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:10:44.680617 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 277/542] 10.222.0.58 () {56 vars in 1102 bytes} [Tue May 17 19:10:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 17 19:10:44.680617 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2624860d-2087-4598-8835-792491f88e3d None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:44.693704 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46df3ca7-ff33-44d0-812b-00b337c912e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:44.694363 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46df3ca7-ff33-44d0-812b-00b337c912e0 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:44.694637 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46df3ca7-ff33-44d0-812b-00b337c912e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:44.694905 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46df3ca7-ff33-44d0-812b-00b337c912e0 None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:44.695718 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46df3ca7-ff33-44d0-812b-00b337c912e0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:44.696020 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46df3ca7-ff33-44d0-812b-00b337c912e0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:44.697456 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46df3ca7-ff33-44d0-812b-00b337c912e0 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:44.710315 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 266/543] 10.222.0.58 () {56 vars in 1148 bytes} [Tue May 17 19:10:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:10:44.721034 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46df3ca7-ff33-44d0-812b-00b337c912e0 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:44.799385 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2624860d-2087-4598-8835-792491f88e3d None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:44.844209 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b108c8f-51fc-4105-8ef6-0644661ec868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:44.844882 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b108c8f-51fc-4105-8ef6-0644661ec868 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:44.845158 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b108c8f-51fc-4105-8ef6-0644661ec868 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:44.845427 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b108c8f-51fc-4105-8ef6-0644661ec868 None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:44.846170 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b108c8f-51fc-4105-8ef6-0644661ec868 None admin] RBAC: Adding query filter params (name=service) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:44.850447 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b108c8f-51fc-4105-8ef6-0644661ec868 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:44.855829 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b108c8f-51fc-4105-8ef6-0644661ec868 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:44.857452 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46df3ca7-ff33-44d0-812b-00b337c912e0 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:44.876061 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 278/544] 10.222.0.58 () {56 vars in 1103 bytes} [Tue May 17 19:10:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:10:44.881529 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b108c8f-51fc-4105-8ef6-0644661ec868 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:44.914737 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6773e8-9835-42f7-8769-3d15f393aede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:44.915389 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6773e8-9835-42f7-8769-3d15f393aede None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:44.915686 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6773e8-9835-42f7-8769-3d15f393aede None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:44.915956 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6773e8-9835-42f7-8769-3d15f393aede None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:44.916780 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6773e8-9835-42f7-8769-3d15f393aede None admin] RBAC: Adding query filter params (role.id=1754a95b778c4e20b53b20cb52ada92b, scope.project.id=1a9d3a0631284c9fb113266036993e2d, user.id=1ac193d15f48463c98ff1d97770ed794) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:44.917141 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6773e8-9835-42f7-8769-3d15f393aede None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:44.918692 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6773e8-9835-42f7-8769-3d15f393aede None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:44.924551 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 267/545] 10.222.0.58 () {56 vars in 1374 bytes} [Tue May 17 19:10:44 2022] GET /identity/v3/role_assignments?user.id=1ac193d15f48463c98ff1d97770ed794&scope.project.id=1a9d3a0631284c9fb113266036993e2d&role.id=1754a95b778c4e20b53b20cb52ada92b => generated 259 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:45.010393 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b108c8f-51fc-4105-8ef6-0644661ec868 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:45.045304 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bdeb99-2a32-4dea-9858-d8737dea06e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:45.046109 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bdeb99-2a32-4dea-9858-d8737dea06e4 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:45.046388 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bdeb99-2a32-4dea-9858-d8737dea06e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:45.046658 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bdeb99-2a32-4dea-9858-d8737dea06e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:45.058639 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bdeb99-2a32-4dea-9858-d8737dea06e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:45.060064 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bdeb99-2a32-4dea-9858-d8737dea06e4 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:45.078669 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-22bdeb99-2a32-4dea-9858-d8737dea06e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:10:45.078669 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:45.078669 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:45.078669 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:45.078669 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:45.078669 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:45.078669 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:45.078669 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:45.078669 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:45.078669 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:45.078669 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:45.078669 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:45.078669 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 17 19:10:45.078669 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 17 19:10:45.078669 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 17 19:10:45.080829 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 17 19:10:45.080829 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:45.080829 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:45.080829 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:45.080829 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:45.080829 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:45.080829 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:45.080829 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:45.080829 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:10:45.080829 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:45.080829 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:45.080829 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:45.080829 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:45.080829 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:45.080829 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:45.080829 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:45.080829 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:45.083468 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 17 19:10:45.083468 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 17 19:10:45.083468 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:10:45.083468 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:10:45.083468 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 279/546] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:10:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 17 19:10:45.091517 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6773e8-9835-42f7-8769-3d15f393aede None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:45.212487 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6773e8-9835-42f7-8769-3d15f393aede None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:45.247003 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfbcd6a-f8e8-4527-8703-7c7fee3390ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:45.247688 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfbcd6a-f8e8-4527-8703-7c7fee3390ac None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:45.247965 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfbcd6a-f8e8-4527-8703-7c7fee3390ac None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:45.248233 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfbcd6a-f8e8-4527-8703-7c7fee3390ac None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:45.248947 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfbcd6a-f8e8-4527-8703-7c7fee3390ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:45.249246 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfbcd6a-f8e8-4527-8703-7c7fee3390ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:45.250613 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfbcd6a-f8e8-4527-8703-7c7fee3390ac None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:45.262630 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 268/547] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 17 19:10:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:45.267959 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bdeb99-2a32-4dea-9858-d8737dea06e4 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:45.370766 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bdeb99-2a32-4dea-9858-d8737dea06e4 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:45.405914 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c14957b-94d5-4d18-9e83-3ff1a7be340f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:45.406706 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c14957b-94d5-4d18-9e83-3ff1a7be340f None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:45.406987 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c14957b-94d5-4d18-9e83-3ff1a7be340f None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:45.407256 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c14957b-94d5-4d18-9e83-3ff1a7be340f None admin] PATH_INFO: `/v3/users/swift` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:45.416460 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c14957b-94d5-4d18-9e83-3ff1a7be340f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:45.418001 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c14957b-94d5-4d18-9e83-3ff1a7be340f None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:45.432332 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-7c14957b-94d5-4d18-9e83-3ff1a7be340f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 17 19:10:45.432332 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:45.432332 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:45.432332 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:45.432332 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:45.432332 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:45.432332 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:45.432332 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:45.432332 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:45.432332 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:45.432332 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:45.432332 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:45.432332 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 17 19:10:45.432332 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_user(user_id) May 17 19:10:45.432332 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 17 19:10:45.432332 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 17 19:10:45.434468 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:45.434468 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:45.434468 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 17 19:10:45.434468 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 17 19:10:45.434468 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 17 19:10:45.434468 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 17 19:10:45.434468 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:45.434468 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:45.434468 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:45.434468 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:45.434468 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:45.434468 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:10:45.434468 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:45.434468 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:45.434468 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:45.434468 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:45.434468 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:45.434468 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:45.436705 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:45.436705 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:45.436705 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 17 19:10:45.436705 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 17 19:10:45.436705 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 17 19:10:45.436705 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 17 19:10:45.436705 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 17 19:10:45.436705 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 17 19:10:45.436705 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 17 19:10:45.436705 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:10:45.436705 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 280/548] 10.222.0.58 () {56 vars in 1096 bytes} [Tue May 17 19:10:45 2022] GET /identity/v3/users/swift => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:10:45.443509 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfbcd6a-f8e8-4527-8703-7c7fee3390ac None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:45.539614 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfbcd6a-f8e8-4527-8703-7c7fee3390ac None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:45.572851 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fc835c-9376-4fb6-a2f2-c349383a7a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:45.573521 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fc835c-9376-4fb6-a2f2-c349383a7a88 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:45.573796 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fc835c-9376-4fb6-a2f2-c349383a7a88 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:45.574068 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fc835c-9376-4fb6-a2f2-c349383a7a88 None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:45.574894 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fc835c-9376-4fb6-a2f2-c349383a7a88 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:45.575197 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fc835c-9376-4fb6-a2f2-c349383a7a88 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:45.576651 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fc835c-9376-4fb6-a2f2-c349383a7a88 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:45.594665 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 269/549] 10.222.0.58 () {56 vars in 1135 bytes} [Tue May 17 19:10:45 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:45.600324 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c14957b-94d5-4d18-9e83-3ff1a7be340f None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:45.698788 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c14957b-94d5-4d18-9e83-3ff1a7be340f None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:45.732387 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2e6f48-7802-4731-92bd-41b780ff392c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:45.733038 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2e6f48-7802-4731-92bd-41b780ff392c None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:45.733316 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2e6f48-7802-4731-92bd-41b780ff392c None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:45.733588 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2e6f48-7802-4731-92bd-41b780ff392c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:45.820133 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2e6f48-7802-4731-92bd-41b780ff392c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:45.821585 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2e6f48-7802-4731-92bd-41b780ff392c None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:45.925104 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 281/550] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 17 19:10:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 17 19:10:45.930683 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fc835c-9376-4fb6-a2f2-c349383a7a88 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:46.032478 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fc835c-9376-4fb6-a2f2-c349383a7a88 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:46.066106 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11331074-33aa-493c-aee1-27b1573954fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:46.066895 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11331074-33aa-493c-aee1-27b1573954fd None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:46.067586 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11331074-33aa-493c-aee1-27b1573954fd None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:46.067858 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11331074-33aa-493c-aee1-27b1573954fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:46.078084 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11331074-33aa-493c-aee1-27b1573954fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:46.079548 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11331074-33aa-493c-aee1-27b1573954fd None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:46.097949 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-11331074-33aa-493c-aee1-27b1573954fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:10:46.097949 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:46.097949 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:46.097949 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:46.097949 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:46.097949 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:46.097949 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:46.097949 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:46.097949 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:46.097949 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:46.097949 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:46.097949 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:46.097949 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 17 19:10:46.097949 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 17 19:10:46.097949 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 17 19:10:46.099051 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 17 19:10:46.099051 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:46.099051 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:46.099051 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:46.099051 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:46.099051 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:46.099051 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:46.099051 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:46.099051 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:10:46.099051 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:46.099051 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:46.099051 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:46.099051 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:46.099051 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:46.099051 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:46.099051 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:46.099051 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:46.100023 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 17 19:10:46.100023 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 17 19:10:46.100023 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:10:46.100023 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:10:46.101806 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 270/551] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:10:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:10:46.107270 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2e6f48-7802-4731-92bd-41b780ff392c None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:46.202419 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f31793e-9b54-4021-bf1d-25945856f86c None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:46.202923 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f31793e-9b54-4021-bf1d-25945856f86c None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:46.203355 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f31793e-9b54-4021-bf1d-25945856f86c None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:46.204857 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 271/552] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:10:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 17 19:10:46.210664 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81170a26-64a8-450c-bbea-e0f281e3a1f2 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:46.211284 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81170a26-64a8-450c-bbea-e0f281e3a1f2 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:46.211773 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81170a26-64a8-450c-bbea-e0f281e3a1f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:46.232090 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2e6f48-7802-4731-92bd-41b780ff392c None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:46.244540 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-81170a26-64a8-450c-bbea-e0f281e3a1f2 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:10:46.245795 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-81170a26-64a8-450c-bbea-e0f281e3a1f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:10:46.266754 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86628557-47c5-4d63-bc4e-5d6668ecdb3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:46.267698 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86628557-47c5-4d63-bc4e-5d6668ecdb3f None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:46.268137 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86628557-47c5-4d63-bc4e-5d6668ecdb3f None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:46.268556 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86628557-47c5-4d63-bc4e-5d6668ecdb3f None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:46.269409 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86628557-47c5-4d63-bc4e-5d6668ecdb3f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:46.269867 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86628557-47c5-4d63-bc4e-5d6668ecdb3f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:46.271935 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86628557-47c5-4d63-bc4e-5d6668ecdb3f None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:46.283778 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 282/553] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 17 19:10:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:46.289873 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86628557-47c5-4d63-bc4e-5d6668ecdb3f None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:46.354179 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 272/554] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:10:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 17 19:10:46.389949 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86628557-47c5-4d63-bc4e-5d6668ecdb3f None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:46.392549 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81170a26-64a8-450c-bbea-e0f281e3a1f2 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:46.423390 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b09020d-594a-43f8-b08e-ff4dcffa7816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:46.424467 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b09020d-594a-43f8-b08e-ff4dcffa7816 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:46.424896 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b09020d-594a-43f8-b08e-ff4dcffa7816 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:46.425305 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b09020d-594a-43f8-b08e-ff4dcffa7816 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:46.432159 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b09020d-594a-43f8-b08e-ff4dcffa7816 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:46.433773 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b09020d-594a-43f8-b08e-ff4dcffa7816 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:46.448504 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-5b09020d-594a-43f8-b08e-ff4dcffa7816 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 17 19:10:46.448504 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:46.448504 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:46.448504 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:46.448504 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:46.448504 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:46.448504 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:46.448504 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:46.448504 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:46.448504 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:46.448504 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:46.448504 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:46.448504 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 17 19:10:46.448504 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(project_id) May 17 19:10:46.448504 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 17 19:10:46.449504 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 17 19:10:46.449504 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:46.449504 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:46.449504 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:46.449504 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:46.449504 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:46.449504 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:46.449504 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:46.449504 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:10:46.449504 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:46.449504 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:46.449504 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:46.449504 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:46.449504 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:46.449504 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:46.449504 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:46.449504 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:46.450429 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 17 19:10:46.450429 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 17 19:10:46.450429 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 17 19:10:46.450429 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 17 19:10:46.450429 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 17 19:10:46.450429 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 17 19:10:46.450429 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 17 19:10:46.450429 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:10:46.452383 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 283/555] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 17 19:10:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 17 19:10:46.459707 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b09020d-594a-43f8-b08e-ff4dcffa7816 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:46.573184 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81170a26-64a8-450c-bbea-e0f281e3a1f2 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:46.642096 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b09020d-594a-43f8-b08e-ff4dcffa7816 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:46.644656 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8230b0-fd89-43d0-8305-6539800c4c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:46.645981 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8230b0-fd89-43d0-8305-6539800c4c27 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:46.646256 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8230b0-fd89-43d0-8305-6539800c4c27 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:46.646523 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8230b0-fd89-43d0-8305-6539800c4c27 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:46.667561 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8230b0-fd89-43d0-8305-6539800c4c27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 17 19:10:46.677498 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-aa8230b0-fd89-43d0-8305-6539800c4c27 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 17 19:10:46.677498 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:46.677498 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:46.677498 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:46.677498 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:46.677498 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:46.677498 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:46.677498 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:46.677498 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:46.677498 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:46.677498 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:46.677498 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:46.677498 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 17 19:10:46.677498 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) May 17 19:10:46.677498 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 17 19:10:46.677498 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 17 19:10:46.678438 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 17 19:10:46.678438 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 17 19:10:46.678438 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 17 19:10:46.678438 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 17 19:10:46.678438 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:46.678438 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:46.678438 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:46.678438 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:46.678438 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:46.678438 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:46.678438 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:46.678438 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:10:46.678438 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:46.678438 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:46.678438 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:46.678438 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:46.679223 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:46.679223 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:46.679223 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:46.679223 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:46.679223 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 17 19:10:46.679223 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 17 19:10:46.679223 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 17 19:10:46.679223 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 17 19:10:46.679223 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 17 19:10:46.679223 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 17 19:10:46.679223 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 17 19:10:46.679223 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:10:46.679223 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 273/556] 10.222.0.58 () {56 vars in 1102 bytes} [Tue May 17 19:10:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:10:46.684076 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8230b0-fd89-43d0-8305-6539800c4c27 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:46.701701 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442b92ea-66b0-4be0-92d6-021251f3b023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:46.703063 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442b92ea-66b0-4be0-92d6-021251f3b023 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:46.703606 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442b92ea-66b0-4be0-92d6-021251f3b023 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:46.704077 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442b92ea-66b0-4be0-92d6-021251f3b023 None admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:46.705078 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442b92ea-66b0-4be0-92d6-021251f3b023 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:46.705579 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442b92ea-66b0-4be0-92d6-021251f3b023 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:46.707241 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442b92ea-66b0-4be0-92d6-021251f3b023 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:46.720272 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 284/557] 10.222.0.58 () {56 vars in 1148 bytes} [Tue May 17 19:10:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:46.727374 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19466f02-0e52-4dee-88c2-f77abad84080 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:46.727943 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19466f02-0e52-4dee-88c2-f77abad84080 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:46.728430 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19466f02-0e52-4dee-88c2-f77abad84080 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:46.730221 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 285/558] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:10:46 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:10:46.731198 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19466f02-0e52-4dee-88c2-f77abad84080 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:46.790017 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8230b0-fd89-43d0-8305-6539800c4c27 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:46.826533 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c29159-ed64-43ff-b9cb-4b5bc6e9d717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:46.827201 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c29159-ed64-43ff-b9cb-4b5bc6e9d717 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:46.827477 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c29159-ed64-43ff-b9cb-4b5bc6e9d717 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:46.827922 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c29159-ed64-43ff-b9cb-4b5bc6e9d717 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:46.828597 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c29159-ed64-43ff-b9cb-4b5bc6e9d717 None admin] RBAC: Adding query filter params (name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:46.828900 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c29159-ed64-43ff-b9cb-4b5bc6e9d717 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:46.830281 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c29159-ed64-43ff-b9cb-4b5bc6e9d717 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:46.842032 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19466f02-0e52-4dee-88c2-f77abad84080 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:46.845159 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 274/559] 10.222.0.58 () {56 vars in 1103 bytes} [Tue May 17 19:10:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:10:46.846309 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b783920-cf79-4084-bb6e-2947bdbc36f2 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:46.846851 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b783920-cf79-4084-bb6e-2947bdbc36f2 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:46.847126 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b783920-cf79-4084-bb6e-2947bdbc36f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:46.864415 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-1b783920-cf79-4084-bb6e-2947bdbc36f2 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:10:46.865606 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-1b783920-cf79-4084-bb6e-2947bdbc36f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:10:46.884357 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b012ecec-2840-4e51-be4c-086f3617a40b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:46.885468 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b012ecec-2840-4e51-be4c-086f3617a40b None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:46.885958 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b012ecec-2840-4e51-be4c-086f3617a40b None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:46.886863 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b012ecec-2840-4e51-be4c-086f3617a40b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:46.888197 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b012ecec-2840-4e51-be4c-086f3617a40b None admin] RBAC: Adding query filter params (role.id=1754a95b778c4e20b53b20cb52ada92b, scope.project.id=1a9d3a0631284c9fb113266036993e2d, user.id=b0cb292002434d30bc81f370d120a2a2) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:46.888865 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b012ecec-2840-4e51-be4c-086f3617a40b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:46.891118 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b012ecec-2840-4e51-be4c-086f3617a40b None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:46.905176 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 286/560] 10.222.0.58 () {56 vars in 1374 bytes} [Tue May 17 19:10:46 2022] GET /identity/v3/role_assignments?user.id=b0cb292002434d30bc81f370d120a2a2&scope.project.id=1a9d3a0631284c9fb113266036993e2d&role.id=1754a95b778c4e20b53b20cb52ada92b => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:46.905176 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b012ecec-2840-4e51-be4c-086f3617a40b None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:46.968758 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 275/561] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:10:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 17 19:10:46.992409 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ade37ae-eb37-467d-bf4b-c00cc310eb7f None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:46.992647 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ade37ae-eb37-467d-bf4b-c00cc310eb7f None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:46.992887 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ade37ae-eb37-467d-bf4b-c00cc310eb7f None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:46.994510 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 276/562] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:10:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 17 19:10:46.995221 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ade37ae-eb37-467d-bf4b-c00cc310eb7f None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:47.064773 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b012ecec-2840-4e51-be4c-086f3617a40b None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:47.103121 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7236c9e9-93d3-4c7c-99f5-e524c92839d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:47.103121 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7236c9e9-93d3-4c7c-99f5-e524c92839d6 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:47.103121 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7236c9e9-93d3-4c7c-99f5-e524c92839d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:47.103478 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7236c9e9-93d3-4c7c-99f5-e524c92839d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:47.116059 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7236c9e9-93d3-4c7c-99f5-e524c92839d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:47.117846 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7236c9e9-93d3-4c7c-99f5-e524c92839d6 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:47.135403 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-7236c9e9-93d3-4c7c-99f5-e524c92839d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:10:47.135403 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:47.135403 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:47.135403 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:47.135403 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:47.135403 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:47.135403 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:47.135403 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:47.135403 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:47.135403 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:47.135403 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:47.135403 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:47.135403 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 17 19:10:47.135403 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 17 19:10:47.135403 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 17 19:10:47.136393 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 17 19:10:47.136393 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:47.136393 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:47.136393 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:47.136393 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:47.136393 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:47.136393 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:47.136393 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:47.136393 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:10:47.136393 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:47.136393 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:47.136393 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:47.136393 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:47.136393 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:47.136393 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:47.136393 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:47.136393 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:47.137828 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 17 19:10:47.137828 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 17 19:10:47.137828 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:10:47.137828 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:10:47.139187 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ade37ae-eb37-467d-bf4b-c00cc310eb7f None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:47.140118 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 287/563] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:10:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:10:47.141537 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91a591b-39be-4b67-8470-521abe64e3d1 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:47.142044 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91a591b-39be-4b67-8470-521abe64e3d1 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:47.142525 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91a591b-39be-4b67-8470-521abe64e3d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:47.159260 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-d91a591b-39be-4b67-8470-521abe64e3d1 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:10:47.161355 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-d91a591b-39be-4b67-8470-521abe64e3d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:10:47.184310 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9a299a-c081-4049-ab18-3fe5b09d9466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:47.185587 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9a299a-c081-4049-ab18-3fe5b09d9466 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:47.185940 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9a299a-c081-4049-ab18-3fe5b09d9466 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:47.186222 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9a299a-c081-4049-ab18-3fe5b09d9466 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:47.199868 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9a299a-c081-4049-ab18-3fe5b09d9466 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 17 19:10:47.209941 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-1e9a299a-c081-4049-ab18-3fe5b09d9466 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 17 19:10:47.209941 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:47.209941 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:47.209941 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:47.209941 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:47.209941 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:47.209941 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:47.209941 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:47.209941 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:47.209941 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:47.209941 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:47.209941 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:47.209941 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 17 19:10:47.209941 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) May 17 19:10:47.209941 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 17 19:10:47.209941 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 17 19:10:47.210877 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 17 19:10:47.210877 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 17 19:10:47.210877 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 17 19:10:47.210877 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 17 19:10:47.210877 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:47.210877 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:47.210877 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:47.210877 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:47.210877 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:47.210877 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:47.210877 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:47.210877 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:10:47.210877 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:47.210877 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:47.210877 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:47.210877 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:47.212202 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:47.212202 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:47.212202 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:47.212202 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:47.212202 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 17 19:10:47.212202 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 17 19:10:47.212202 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 17 19:10:47.212202 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 17 19:10:47.212202 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 17 19:10:47.212202 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 17 19:10:47.212202 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 17 19:10:47.212202 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:10:47.212202 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 277/564] 10.222.0.58 () {56 vars in 1102 bytes} [Tue May 17 19:10:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:10:47.212202 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9a299a-c081-4049-ab18-3fe5b09d9466 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:47.242100 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 288/565] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:10:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 17 19:10:47.243676 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91a591b-39be-4b67-8470-521abe64e3d1 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:47.341245 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9a299a-c081-4049-ab18-3fe5b09d9466 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:47.342897 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91a591b-39be-4b67-8470-521abe64e3d1 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:47.377361 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f918d127-bde1-469c-9630-dc8cadc58194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:47.378392 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f918d127-bde1-469c-9630-dc8cadc58194 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:47.378841 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f918d127-bde1-469c-9630-dc8cadc58194 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:47.379341 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f918d127-bde1-469c-9630-dc8cadc58194 None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:47.380874 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f918d127-bde1-469c-9630-dc8cadc58194 None admin] RBAC: Adding query filter params (name=service) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:47.381368 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f918d127-bde1-469c-9630-dc8cadc58194 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:47.382879 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d8c9e1-3bb8-4abf-be70-c91bf246feea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:47.383133 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f918d127-bde1-469c-9630-dc8cadc58194 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:47.383694 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d8c9e1-3bb8-4abf-be70-c91bf246feea None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:47.384040 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d8c9e1-3bb8-4abf-be70-c91bf246feea None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:47.384384 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d8c9e1-3bb8-4abf-be70-c91bf246feea None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:47.385340 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d8c9e1-3bb8-4abf-be70-c91bf246feea None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:47.385926 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d8c9e1-3bb8-4abf-be70-c91bf246feea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:47.387997 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d8c9e1-3bb8-4abf-be70-c91bf246feea None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:47.398421 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 289/566] 10.222.0.58 () {56 vars in 1103 bytes} [Tue May 17 19:10:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:47.399880 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f918d127-bde1-469c-9630-dc8cadc58194 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:47.410173 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 278/567] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 17 19:10:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:47.411367 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d8c9e1-3bb8-4abf-be70-c91bf246feea None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:47.507554 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f918d127-bde1-469c-9630-dc8cadc58194 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:47.538339 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d8c9e1-3bb8-4abf-be70-c91bf246feea None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:47.539581 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e603f22-6861-4e6f-87a8-2f9536bebcee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:47.540459 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e603f22-6861-4e6f-87a8-2f9536bebcee None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:47.540866 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e603f22-6861-4e6f-87a8-2f9536bebcee None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:47.541141 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e603f22-6861-4e6f-87a8-2f9536bebcee None admin] PATH_INFO: `/v3/roles/service` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:47.552027 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e603f22-6861-4e6f-87a8-2f9536bebcee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 17 19:10:47.559521 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-1e603f22-6861-4e6f-87a8-2f9536bebcee None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 17 19:10:47.559521 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:47.559521 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:47.559521 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:47.559521 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:47.559521 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:47.559521 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:47.559521 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:47.559521 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:47.559521 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:47.559521 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:47.559521 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:47.559521 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 17 19:10:47.559521 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) May 17 19:10:47.559521 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 17 19:10:47.559521 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 17 19:10:47.560625 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 17 19:10:47.560625 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 17 19:10:47.560625 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 17 19:10:47.560625 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 17 19:10:47.560625 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:47.560625 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:47.560625 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:47.560625 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:47.560625 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:47.560625 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:47.560625 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:47.560625 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:10:47.560625 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:47.560625 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:47.560625 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:47.560625 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:47.561804 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:47.561804 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:47.561804 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:47.561804 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:47.561804 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 17 19:10:47.561804 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 17 19:10:47.561804 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 17 19:10:47.561804 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 17 19:10:47.561804 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 17 19:10:47.561804 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 17 19:10:47.561804 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 17 19:10:47.561804 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:10:47.565404 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 290/568] 10.222.0.58 () {56 vars in 1102 bytes} [Tue May 17 19:10:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:10:47.566235 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e603f22-6861-4e6f-87a8-2f9536bebcee None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:47.578939 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dc1ea4-a4fd-4cd1-90f7-30a97f1a35d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:47.580065 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dc1ea4-a4fd-4cd1-90f7-30a97f1a35d5 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:47.580505 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dc1ea4-a4fd-4cd1-90f7-30a97f1a35d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:47.580926 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dc1ea4-a4fd-4cd1-90f7-30a97f1a35d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:47.592447 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dc1ea4-a4fd-4cd1-90f7-30a97f1a35d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:47.594224 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dc1ea4-a4fd-4cd1-90f7-30a97f1a35d5 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:47.613636 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-54dc1ea4-a4fd-4cd1-90f7-30a97f1a35d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:10:47.613636 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:47.613636 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:47.613636 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:47.613636 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:47.613636 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:47.613636 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:47.613636 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:47.613636 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:47.613636 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:47.613636 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:47.613636 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:47.613636 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 17 19:10:47.613636 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 17 19:10:47.613636 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 17 19:10:47.615245 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 17 19:10:47.615245 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:47.615245 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:47.615245 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:47.615245 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:47.615245 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:47.615245 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:47.615245 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:47.615245 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:10:47.615245 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:47.615245 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:47.615245 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:47.615245 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:47.615245 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:47.615245 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:47.615245 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:47.615245 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:47.616152 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 17 19:10:47.616152 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 17 19:10:47.616152 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:10:47.616152 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:10:47.618058 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 279/569] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:10:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:10:47.619171 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dc1ea4-a4fd-4cd1-90f7-30a97f1a35d5 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:47.682101 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e603f22-6861-4e6f-87a8-2f9536bebcee None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:47.739457 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0fdad5-281e-4c52-bf60-129dc09aa7db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:47.740262 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0fdad5-281e-4c52-bf60-129dc09aa7db None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:47.740540 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0fdad5-281e-4c52-bf60-129dc09aa7db None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:47.740807 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0fdad5-281e-4c52-bf60-129dc09aa7db None admin] PATH_INFO: `/v3/users/nova` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:47.750129 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0fdad5-281e-4c52-bf60-129dc09aa7db None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:47.751987 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0fdad5-281e-4c52-bf60-129dc09aa7db None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:47.770539 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-3c0fdad5-281e-4c52-bf60-129dc09aa7db None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 17 19:10:47.770539 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:47.770539 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:47.770539 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:47.770539 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:47.770539 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:47.770539 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:47.770539 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:47.770539 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:47.770539 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:47.770539 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:47.770539 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:47.770539 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 17 19:10:47.770539 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_user(user_id) May 17 19:10:47.770539 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 17 19:10:47.770539 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 17 19:10:47.771544 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:47.771544 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:47.771544 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 17 19:10:47.771544 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 17 19:10:47.771544 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 17 19:10:47.771544 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 17 19:10:47.771544 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:47.771544 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:47.771544 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:47.771544 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:47.771544 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:47.771544 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:10:47.771544 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:47.771544 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:47.771544 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:47.771544 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:47.771544 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:47.771544 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:47.773588 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:47.773588 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:47.773588 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 17 19:10:47.773588 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 17 19:10:47.773588 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 17 19:10:47.773588 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 17 19:10:47.773588 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 17 19:10:47.773588 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 17 19:10:47.773588 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 17 19:10:47.773588 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:10:47.773588 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 291/570] 10.222.0.58 () {56 vars in 1093 bytes} [Tue May 17 19:10:47 2022] GET /identity/v3/users/nova => generated 82 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:10:47.773588 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0fdad5-281e-4c52-bf60-129dc09aa7db None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:47.781885 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dc1ea4-a4fd-4cd1-90f7-30a97f1a35d5 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:47.822410 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13363a85-f14c-49d7-b957-aeb843008e59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:47.823419 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13363a85-f14c-49d7-b957-aeb843008e59 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:47.823731 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13363a85-f14c-49d7-b957-aeb843008e59 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:47.824003 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13363a85-f14c-49d7-b957-aeb843008e59 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:47.824740 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13363a85-f14c-49d7-b957-aeb843008e59 None admin] RBAC: Adding query filter params (name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:47.825042 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13363a85-f14c-49d7-b957-aeb843008e59 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:47.826410 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13363a85-f14c-49d7-b957-aeb843008e59 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:47.839130 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 280/571] 10.222.0.58 () {56 vars in 1103 bytes} [Tue May 17 19:10:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:47.839130 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13363a85-f14c-49d7-b957-aeb843008e59 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:47.881017 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0fdad5-281e-4c52-bf60-129dc09aa7db None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:47.916146 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef31b948-4581-41b4-8521-cff8d38e0c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:47.916809 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef31b948-4581-41b4-8521-cff8d38e0c30 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:47.917087 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef31b948-4581-41b4-8521-cff8d38e0c30 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:47.917358 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef31b948-4581-41b4-8521-cff8d38e0c30 None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:47.918071 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef31b948-4581-41b4-8521-cff8d38e0c30 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:47.918370 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef31b948-4581-41b4-8521-cff8d38e0c30 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:47.920157 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef31b948-4581-41b4-8521-cff8d38e0c30 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:47.938723 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 292/572] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 17 19:10:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:47.938723 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571b481b-023b-449c-b1f4-77fd4311f66d None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:47.938723 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571b481b-023b-449c-b1f4-77fd4311f66d None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:47.938723 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571b481b-023b-449c-b1f4-77fd4311f66d None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:47.939808 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 293/573] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:10:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:10:47.941354 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571b481b-023b-449c-b1f4-77fd4311f66d None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:47.943706 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13363a85-f14c-49d7-b957-aeb843008e59 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:47.979071 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea829ec-6679-44f2-9a20-7ec6f9849018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:47.980448 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea829ec-6679-44f2-9a20-7ec6f9849018 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:47.980926 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea829ec-6679-44f2-9a20-7ec6f9849018 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:47.981385 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea829ec-6679-44f2-9a20-7ec6f9849018 None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:47.982379 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea829ec-6679-44f2-9a20-7ec6f9849018 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:47.982866 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea829ec-6679-44f2-9a20-7ec6f9849018 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:47.984893 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea829ec-6679-44f2-9a20-7ec6f9849018 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:48.006144 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 281/574] 10.222.0.58 () {56 vars in 1133 bytes} [Tue May 17 19:10:47 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:48.007112 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea829ec-6679-44f2-9a20-7ec6f9849018 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:48.051822 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571b481b-023b-449c-b1f4-77fd4311f66d None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:48.085886 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2ff698-485c-46b3-a7a7-3f19ba73b5b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:48.086671 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2ff698-485c-46b3-a7a7-3f19ba73b5b3 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:48.086945 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2ff698-485c-46b3-a7a7-3f19ba73b5b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:48.087229 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2ff698-485c-46b3-a7a7-3f19ba73b5b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:48.100813 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2ff698-485c-46b3-a7a7-3f19ba73b5b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:48.102691 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2ff698-485c-46b3-a7a7-3f19ba73b5b3 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:48.105865 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea829ec-6679-44f2-9a20-7ec6f9849018 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:48.123409 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-1d2ff698-485c-46b3-a7a7-3f19ba73b5b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:10:48.123409 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:48.123409 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:48.123409 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:48.123409 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:48.123409 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:48.123409 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:48.123409 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:48.123409 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:48.123409 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:48.123409 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:48.123409 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:48.123409 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 17 19:10:48.123409 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 17 19:10:48.123409 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 17 19:10:48.124308 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 17 19:10:48.124308 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:48.124308 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:48.124308 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:48.124308 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:48.124308 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:48.124308 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:48.124308 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:48.124308 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:10:48.124308 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:48.124308 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:48.124308 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:48.124308 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:48.124308 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:48.124308 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:48.124308 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:48.124308 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:48.125181 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 17 19:10:48.125181 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 17 19:10:48.125181 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:10:48.125181 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:10:48.129135 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 294/575] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:10:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:10:48.131075 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8a0c87-98b3-4fb1-ba21-cd147dacdf79 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:48.131356 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8a0c87-98b3-4fb1-ba21-cd147dacdf79 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:48.131654 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8a0c87-98b3-4fb1-ba21-cd147dacdf79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:48.139872 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4b66d7-9597-4ae0-bc1f-12a9ed029d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:48.140662 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4b66d7-9597-4ae0-bc1f-12a9ed029d4f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:48.140937 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4b66d7-9597-4ae0-bc1f-12a9ed029d4f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:48.141206 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4b66d7-9597-4ae0-bc1f-12a9ed029d4f None admin] PATH_INFO: `/v3/users/glance` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:48.148851 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-bc8a0c87-98b3-4fb1-ba21-cd147dacdf79 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:10:48.149183 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4b66d7-9597-4ae0-bc1f-12a9ed029d4f None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:48.150010 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-bc8a0c87-98b3-4fb1-ba21-cd147dacdf79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:10:48.150626 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4b66d7-9597-4ae0-bc1f-12a9ed029d4f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:48.165565 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-ff4b66d7-9597-4ae0-bc1f-12a9ed029d4f None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 17 19:10:48.165565 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:48.165565 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:48.165565 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:48.165565 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:48.165565 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:48.165565 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:48.165565 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:48.165565 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:48.165565 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:48.165565 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:48.165565 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:48.165565 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 17 19:10:48.165565 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_user(user_id) May 17 19:10:48.165565 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 17 19:10:48.165565 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 17 19:10:48.166533 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:48.166533 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:48.166533 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 17 19:10:48.166533 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 17 19:10:48.166533 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 17 19:10:48.166533 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 17 19:10:48.166533 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:48.166533 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:48.166533 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:48.166533 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:48.166533 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:48.166533 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:10:48.166533 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:48.166533 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:48.166533 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:48.166533 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:48.166533 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:48.166533 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:48.167471 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:48.167471 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:48.167471 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 17 19:10:48.167471 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 17 19:10:48.167471 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 17 19:10:48.167471 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 17 19:10:48.167471 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 17 19:10:48.167471 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 17 19:10:48.167471 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 17 19:10:48.167471 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:10:48.167471 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 282/576] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 17 19:10:48 2022] GET /identity/v3/users/glance => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:10:48.168105 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4b66d7-9597-4ae0-bc1f-12a9ed029d4f None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:48.228113 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 295/577] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:10:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 17 19:10:48.228956 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8a0c87-98b3-4fb1-ba21-cd147dacdf79 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:48.285634 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4b66d7-9597-4ae0-bc1f-12a9ed029d4f None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:48.318512 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd7005c-0969-4836-bf3c-38b0e234dd34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:48.319223 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd7005c-0969-4836-bf3c-38b0e234dd34 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:48.319520 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd7005c-0969-4836-bf3c-38b0e234dd34 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:48.319787 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd7005c-0969-4836-bf3c-38b0e234dd34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:48.330608 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd7005c-0969-4836-bf3c-38b0e234dd34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:48.331187 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8a0c87-98b3-4fb1-ba21-cd147dacdf79 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:48.332132 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd7005c-0969-4836-bf3c-38b0e234dd34 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:48.350288 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-3dd7005c-0969-4836-bf3c-38b0e234dd34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:10:48.350288 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:48.350288 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:48.350288 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:48.350288 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:48.350288 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:48.350288 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:48.350288 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:48.350288 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:48.350288 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:48.350288 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:48.350288 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:48.350288 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 17 19:10:48.350288 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 17 19:10:48.350288 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 17 19:10:48.351209 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 17 19:10:48.351209 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:48.351209 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:48.351209 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:48.351209 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:48.351209 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:48.351209 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:48.351209 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:48.351209 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:10:48.351209 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:48.351209 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:48.351209 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:48.351209 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:48.351209 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:48.351209 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:48.351209 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:48.351209 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:48.352118 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 17 19:10:48.352118 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 17 19:10:48.352118 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:10:48.352118 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:10:48.352118 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 283/578] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:10:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:10:48.352960 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd7005c-0969-4836-bf3c-38b0e234dd34 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:48.369810 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9429d1-b200-4e6e-99cf-c0d173b29420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:48.370462 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9429d1-b200-4e6e-99cf-c0d173b29420 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:48.370738 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9429d1-b200-4e6e-99cf-c0d173b29420 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:48.371007 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9429d1-b200-4e6e-99cf-c0d173b29420 None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:48.371747 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9429d1-b200-4e6e-99cf-c0d173b29420 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:48.372044 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9429d1-b200-4e6e-99cf-c0d173b29420 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:48.373420 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9429d1-b200-4e6e-99cf-c0d173b29420 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:48.384722 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 296/579] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 17 19:10:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:48.385330 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9429d1-b200-4e6e-99cf-c0d173b29420 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:48.457920 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd7005c-0969-4836-bf3c-38b0e234dd34 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:48.483473 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9429d1-b200-4e6e-99cf-c0d173b29420 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:48.490022 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbb06d2-ad31-4d51-ab5c-1638909ef083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:48.490686 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbb06d2-ad31-4d51-ab5c-1638909ef083 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:48.490959 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbb06d2-ad31-4d51-ab5c-1638909ef083 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:48.491228 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbb06d2-ad31-4d51-ab5c-1638909ef083 None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:48.492077 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbb06d2-ad31-4d51-ab5c-1638909ef083 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:48.492384 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbb06d2-ad31-4d51-ab5c-1638909ef083 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:48.493841 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbb06d2-ad31-4d51-ab5c-1638909ef083 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:48.510476 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 284/580] 10.222.0.58 () {56 vars in 1137 bytes} [Tue May 17 19:10:48 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:48.511329 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbb06d2-ad31-4d51-ab5c-1638909ef083 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:48.522322 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42632504-29fd-46a9-a7d7-3c1dcb0a700c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:48.523108 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42632504-29fd-46a9-a7d7-3c1dcb0a700c None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:48.523382 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42632504-29fd-46a9-a7d7-3c1dcb0a700c None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:48.523679 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42632504-29fd-46a9-a7d7-3c1dcb0a700c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:48.533724 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42632504-29fd-46a9-a7d7-3c1dcb0a700c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 17 19:10:48.542218 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-42632504-29fd-46a9-a7d7-3c1dcb0a700c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 17 19:10:48.542218 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:48.542218 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:48.542218 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:48.542218 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:48.542218 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:48.542218 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:48.542218 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:48.542218 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:48.542218 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:48.542218 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:48.542218 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:48.542218 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 17 19:10:48.542218 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) May 17 19:10:48.542218 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 17 19:10:48.542218 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 17 19:10:48.543139 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 17 19:10:48.543139 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 17 19:10:48.543139 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 17 19:10:48.543139 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 17 19:10:48.543139 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:48.543139 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:48.543139 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:48.543139 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:48.543139 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:48.543139 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:48.543139 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:48.543139 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:10:48.543139 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:48.543139 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:48.543139 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:48.543139 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:48.548848 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:48.548848 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:48.548848 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:48.548848 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:48.548848 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 17 19:10:48.548848 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 17 19:10:48.548848 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 17 19:10:48.548848 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 17 19:10:48.548848 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 17 19:10:48.548848 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 17 19:10:48.548848 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 17 19:10:48.548848 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:10:48.548848 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 297/581] 10.222.0.58 () {56 vars in 1102 bytes} [Tue May 17 19:10:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:10:48.548848 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42632504-29fd-46a9-a7d7-3c1dcb0a700c None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:48.613420 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbb06d2-ad31-4d51-ab5c-1638909ef083 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:48.646583 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76ccb5a-22b3-4cca-84cf-c442b7dd5a5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:48.647286 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76ccb5a-22b3-4cca-84cf-c442b7dd5a5b None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:48.647589 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76ccb5a-22b3-4cca-84cf-c442b7dd5a5b None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:48.647858 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76ccb5a-22b3-4cca-84cf-c442b7dd5a5b None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:48.648608 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76ccb5a-22b3-4cca-84cf-c442b7dd5a5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:48.648904 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76ccb5a-22b3-4cca-84cf-c442b7dd5a5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:48.649227 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42632504-29fd-46a9-a7d7-3c1dcb0a700c None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:48.650264 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76ccb5a-22b3-4cca-84cf-c442b7dd5a5b None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:48.661959 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 285/582] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 17 19:10:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:48.662787 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76ccb5a-22b3-4cca-84cf-c442b7dd5a5b None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:48.686978 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3e63f0-8ac7-4d8a-bfb8-0ae47e918691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:48.688082 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3e63f0-8ac7-4d8a-bfb8-0ae47e918691 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:48.688542 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3e63f0-8ac7-4d8a-bfb8-0ae47e918691 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:48.689008 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3e63f0-8ac7-4d8a-bfb8-0ae47e918691 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:48.697241 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3e63f0-8ac7-4d8a-bfb8-0ae47e918691 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:48.698928 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3e63f0-8ac7-4d8a-bfb8-0ae47e918691 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:48.713685 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-ac3e63f0-8ac7-4d8a-bfb8-0ae47e918691 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 17 19:10:48.713685 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:48.713685 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:48.713685 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:48.713685 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:48.713685 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:48.713685 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:48.713685 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:48.713685 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:48.713685 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:48.713685 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:48.713685 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:48.713685 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 17 19:10:48.713685 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_user(user_id) May 17 19:10:48.713685 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 17 19:10:48.713685 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 17 19:10:48.714812 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:48.714812 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:48.714812 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 17 19:10:48.714812 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 17 19:10:48.714812 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 17 19:10:48.714812 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 17 19:10:48.714812 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:48.714812 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:48.714812 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:48.714812 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:48.714812 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:48.714812 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:10:48.714812 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:48.714812 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:48.714812 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:48.714812 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:48.714812 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:48.714812 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:48.716148 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:48.716148 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:48.716148 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 17 19:10:48.716148 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 17 19:10:48.716148 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 17 19:10:48.716148 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 17 19:10:48.716148 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 17 19:10:48.716148 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 17 19:10:48.716148 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 17 19:10:48.716148 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:10:48.721592 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 298/583] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 17 19:10:48 2022] GET /identity/v3/users/cinder => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:10:48.721592 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3e63f0-8ac7-4d8a-bfb8-0ae47e918691 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:48.761862 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76ccb5a-22b3-4cca-84cf-c442b7dd5a5b None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:48.794355 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6b4693-376f-4deb-912e-f88253d046cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:48.795219 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6b4693-376f-4deb-912e-f88253d046cf None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:48.795610 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6b4693-376f-4deb-912e-f88253d046cf None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:48.795947 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6b4693-376f-4deb-912e-f88253d046cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:48.806575 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6b4693-376f-4deb-912e-f88253d046cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:48.808139 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6b4693-376f-4deb-912e-f88253d046cf None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:48.815730 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3e63f0-8ac7-4d8a-bfb8-0ae47e918691 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:48.825289 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-0c6b4693-376f-4deb-912e-f88253d046cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:10:48.825289 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:48.825289 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:48.825289 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:48.825289 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:48.825289 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:48.825289 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:48.825289 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:48.825289 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:48.825289 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:48.825289 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:48.825289 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:48.825289 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 17 19:10:48.825289 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 17 19:10:48.825289 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 17 19:10:48.826458 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 17 19:10:48.826458 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:48.826458 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:48.826458 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:48.826458 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:48.826458 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:48.826458 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:48.826458 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:48.826458 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:10:48.826458 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:48.826458 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:48.826458 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:48.826458 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:48.826458 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:48.826458 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:48.826458 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:48.826458 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:48.827561 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 17 19:10:48.827561 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 17 19:10:48.827561 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:10:48.827561 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:10:48.831063 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 286/584] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:10:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:10:48.831922 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6b4693-376f-4deb-912e-f88253d046cf None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:48.852748 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c3b3d9-e3ce-488a-b33e-0d1b99438a89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:48.853540 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c3b3d9-e3ce-488a-b33e-0d1b99438a89 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:48.853914 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c3b3d9-e3ce-488a-b33e-0d1b99438a89 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:48.854280 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c3b3d9-e3ce-488a-b33e-0d1b99438a89 None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:48.855114 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c3b3d9-e3ce-488a-b33e-0d1b99438a89 None admin] RBAC: Adding query filter params (name=service) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:48.855567 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c3b3d9-e3ce-488a-b33e-0d1b99438a89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:48.857000 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c3b3d9-e3ce-488a-b33e-0d1b99438a89 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:48.866093 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 299/585] 10.222.0.58 () {56 vars in 1103 bytes} [Tue May 17 19:10:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:48.866928 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c3b3d9-e3ce-488a-b33e-0d1b99438a89 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:48.938767 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6b4693-376f-4deb-912e-f88253d046cf None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:48.962563 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c3b3d9-e3ce-488a-b33e-0d1b99438a89 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:48.974975 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eec262-b0d9-4f08-a77f-b72456421d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:48.975909 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eec262-b0d9-4f08-a77f-b72456421d07 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:48.976287 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eec262-b0d9-4f08-a77f-b72456421d07 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:48.976678 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eec262-b0d9-4f08-a77f-b72456421d07 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:48.983387 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5eec262-b0d9-4f08-a77f-b72456421d07 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:48.985622 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5eec262-b0d9-4f08-a77f-b72456421d07 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:48.995380 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8311249-83e6-4a76-9679-c0351b02650e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:48.996101 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8311249-83e6-4a76-9679-c0351b02650e None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:48.996379 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8311249-83e6-4a76-9679-c0351b02650e None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:48.996654 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8311249-83e6-4a76-9679-c0351b02650e None admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:48.997507 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8311249-83e6-4a76-9679-c0351b02650e None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:48.997815 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8311249-83e6-4a76-9679-c0351b02650e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:48.999285 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8311249-83e6-4a76-9679-c0351b02650e None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:49.002553 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-e5eec262-b0d9-4f08-a77f-b72456421d07 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 17 19:10:49.002553 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:49.002553 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:49.002553 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:49.002553 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:49.002553 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:49.002553 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:49.002553 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:49.002553 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:49.002553 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:49.002553 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:49.002553 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:49.002553 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 17 19:10:49.002553 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(project_id) May 17 19:10:49.002553 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 17 19:10:49.003745 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 17 19:10:49.003745 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:49.003745 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:49.003745 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:49.003745 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:49.003745 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:49.003745 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:49.003745 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:49.003745 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:10:49.003745 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:49.003745 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:49.003745 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:49.003745 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:49.003745 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:49.003745 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:49.003745 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:49.003745 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:49.005501 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 17 19:10:49.005501 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 17 19:10:49.005501 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 17 19:10:49.005501 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 17 19:10:49.005501 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 17 19:10:49.005501 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 17 19:10:49.005501 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 17 19:10:49.005501 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:10:49.009338 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 287/586] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 17 19:10:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:10:49.010141 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eec262-b0d9-4f08-a77f-b72456421d07 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:49.023981 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 300/587] 10.222.0.58 () {56 vars in 1137 bytes} [Tue May 17 19:10:48 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:49.025155 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8311249-83e6-4a76-9679-c0351b02650e None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:49.108945 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eec262-b0d9-4f08-a77f-b72456421d07 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:49.122435 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8311249-83e6-4a76-9679-c0351b02650e None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:49.147228 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889060be-99ec-4a84-b51d-4191ae05e6e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:49.147914 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889060be-99ec-4a84-b51d-4191ae05e6e1 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:49.148188 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889060be-99ec-4a84-b51d-4191ae05e6e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:49.148462 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889060be-99ec-4a84-b51d-4191ae05e6e1 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:49.149240 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889060be-99ec-4a84-b51d-4191ae05e6e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:49.149539 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889060be-99ec-4a84-b51d-4191ae05e6e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:49.150897 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889060be-99ec-4a84-b51d-4191ae05e6e1 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:49.156092 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9a1744-bf02-42d5-92a6-3477fc4ffdb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:49.156878 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9a1744-bf02-42d5-92a6-3477fc4ffdb5 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:49.157155 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9a1744-bf02-42d5-92a6-3477fc4ffdb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:49.157424 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9a1744-bf02-42d5-92a6-3477fc4ffdb5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:49.167243 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 288/588] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 17 19:10:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:49.167243 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889060be-99ec-4a84-b51d-4191ae05e6e1 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:49.168065 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9a1744-bf02-42d5-92a6-3477fc4ffdb5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:49.169538 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9a1744-bf02-42d5-92a6-3477fc4ffdb5 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:49.187305 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-9b9a1744-bf02-42d5-92a6-3477fc4ffdb5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:10:49.187305 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:49.187305 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:49.187305 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:49.187305 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:49.187305 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:49.187305 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:49.187305 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:49.187305 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:49.187305 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:49.187305 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:49.187305 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:49.187305 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 17 19:10:49.187305 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 17 19:10:49.187305 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 17 19:10:49.188444 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 17 19:10:49.188444 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:49.188444 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:49.188444 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:49.188444 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:49.188444 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:49.188444 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:49.188444 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:49.188444 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:10:49.188444 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:49.188444 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:49.188444 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:49.188444 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:49.188444 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:49.188444 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:49.188444 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:49.188444 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:49.189357 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 17 19:10:49.189357 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 17 19:10:49.189357 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:10:49.189357 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:10:49.189357 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 301/589] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:10:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:10:49.189684 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9a1744-bf02-42d5-92a6-3477fc4ffdb5 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:49.265397 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889060be-99ec-4a84-b51d-4191ae05e6e1 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:49.282038 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9a1744-bf02-42d5-92a6-3477fc4ffdb5 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:49.296495 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561369bf-a51f-4b38-8d95-74501c59f2e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:49.297347 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561369bf-a51f-4b38-8d95-74501c59f2e1 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:49.297625 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561369bf-a51f-4b38-8d95-74501c59f2e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:49.297892 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561369bf-a51f-4b38-8d95-74501c59f2e1 None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:49.298659 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561369bf-a51f-4b38-8d95-74501c59f2e1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:49.298957 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561369bf-a51f-4b38-8d95-74501c59f2e1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:49.300414 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561369bf-a51f-4b38-8d95-74501c59f2e1 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:49.312366 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 289/590] 10.222.0.58 () {56 vars in 1148 bytes} [Tue May 17 19:10:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:49.313127 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561369bf-a51f-4b38-8d95-74501c59f2e1 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:49.314437 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e7693d-8839-4d6c-b97f-ccdfb3475dfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:49.315215 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e7693d-8839-4d6c-b97f-ccdfb3475dfc None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:49.315515 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e7693d-8839-4d6c-b97f-ccdfb3475dfc None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:49.315789 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e7693d-8839-4d6c-b97f-ccdfb3475dfc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:49.325703 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e7693d-8839-4d6c-b97f-ccdfb3475dfc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:49.327136 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e7693d-8839-4d6c-b97f-ccdfb3475dfc None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:49.344289 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-b0e7693d-8839-4d6c-b97f-ccdfb3475dfc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:10:49.344289 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:49.344289 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:49.344289 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:49.344289 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:49.344289 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:49.344289 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:49.344289 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:49.344289 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:49.344289 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:49.344289 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:49.344289 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:49.344289 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 17 19:10:49.344289 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 17 19:10:49.344289 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 17 19:10:49.345185 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 17 19:10:49.345185 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:49.345185 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:49.345185 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:49.345185 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:49.345185 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:49.345185 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:49.345185 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:49.345185 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:10:49.345185 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:49.345185 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:49.345185 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:49.345185 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:49.345185 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:49.345185 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:49.345185 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:49.345185 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:49.346119 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 17 19:10:49.346119 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 17 19:10:49.346119 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:10:49.346119 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:10:49.346119 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 302/591] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:10:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:10:49.346993 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0048818-cfcb-430e-b374-628af18229df None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:49.347275 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0048818-cfcb-430e-b374-628af18229df None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:49.347569 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0048818-cfcb-430e-b374-628af18229df None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:49.349570 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 303/592] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:10:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:10:49.350170 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0048818-cfcb-430e-b374-628af18229df None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:49.406431 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561369bf-a51f-4b38-8d95-74501c59f2e1 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:49.437624 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72ac6db-fd3b-4c37-b965-762ba7d37f7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:49.438393 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72ac6db-fd3b-4c37-b965-762ba7d37f7f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:49.438667 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72ac6db-fd3b-4c37-b965-762ba7d37f7f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:49.438931 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72ac6db-fd3b-4c37-b965-762ba7d37f7f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:49.445130 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0048818-cfcb-430e-b374-628af18229df None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:49.445301 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72ac6db-fd3b-4c37-b965-762ba7d37f7f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:49.446729 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72ac6db-fd3b-4c37-b965-762ba7d37f7f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:49.461104 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-b72ac6db-fd3b-4c37-b965-762ba7d37f7f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 17 19:10:49.461104 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:49.461104 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:49.461104 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:49.461104 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:49.461104 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:49.461104 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:49.461104 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:49.461104 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:49.461104 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:49.461104 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:49.461104 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:49.461104 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 17 19:10:49.461104 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(project_id) May 17 19:10:49.461104 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 17 19:10:49.461964 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 17 19:10:49.461964 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:49.461964 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:49.461964 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:49.461964 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:49.461964 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:49.461964 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:49.461964 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:49.461964 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:10:49.461964 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:49.461964 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:49.461964 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:49.461964 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:49.461964 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:49.461964 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:49.461964 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:49.461964 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:49.464320 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 17 19:10:49.464320 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 17 19:10:49.464320 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 17 19:10:49.464320 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 17 19:10:49.464320 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 17 19:10:49.464320 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 17 19:10:49.464320 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 17 19:10:49.464320 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:10:49.467719 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 290/593] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 17 19:10:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:10:49.467719 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72ac6db-fd3b-4c37-b965-762ba7d37f7f None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:49.478838 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547535e2-af8c-492f-9f4a-5abe2fb6065d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:49.479504 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547535e2-af8c-492f-9f4a-5abe2fb6065d None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:49.479796 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547535e2-af8c-492f-9f4a-5abe2fb6065d None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:49.480070 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547535e2-af8c-492f-9f4a-5abe2fb6065d None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:49.480778 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547535e2-af8c-492f-9f4a-5abe2fb6065d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:49.481077 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547535e2-af8c-492f-9f4a-5abe2fb6065d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:49.482434 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547535e2-af8c-492f-9f4a-5abe2fb6065d None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:49.493071 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 304/594] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 17 19:10:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:49.493834 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547535e2-af8c-492f-9f4a-5abe2fb6065d None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:49.560236 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72ac6db-fd3b-4c37-b965-762ba7d37f7f None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:49.585064 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547535e2-af8c-492f-9f4a-5abe2fb6065d None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:49.591341 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e779bfb7-21a0-453a-b658-9a09ab547528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:49.592237 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e779bfb7-21a0-453a-b658-9a09ab547528 None admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:49.592510 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e779bfb7-21a0-453a-b658-9a09ab547528 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:49.592777 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e779bfb7-21a0-453a-b658-9a09ab547528 None admin] PATH_INFO: `/v3/projects/1a9d3a0631284c9fb113266036993e2d/users/65c1f878b0dc4d63a4f5dcb12fac1dc1/roles/1754a95b778c4e20b53b20cb52ada92b` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:49.606383 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e779bfb7-21a0-453a-b658-9a09ab547528 None admin] RBAC: Authorizing `identity:create_grant(project_id=1a9d3a0631284c9fb113266036993e2d, user_id=65c1f878b0dc4d63a4f5dcb12fac1dc1, role_id=1754a95b778c4e20b53b20cb52ada92b)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:49.607826 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e779bfb7-21a0-453a-b658-9a09ab547528 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:49.616518 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81f588a-cfc1-403c-b461-c58210c501b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:49.617162 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81f588a-cfc1-403c-b461-c58210c501b7 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:49.617440 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81f588a-cfc1-403c-b461-c58210c501b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:49.617710 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81f588a-cfc1-403c-b461-c58210c501b7 None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:49.618416 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81f588a-cfc1-403c-b461-c58210c501b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:49.618716 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81f588a-cfc1-403c-b461-c58210c501b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:49.620085 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81f588a-cfc1-403c-b461-c58210c501b7 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:49.631760 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 305/595] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 17 19:10:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:49.633029 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35654ce-f0b3-46fe-805f-a0f4f7424814 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:49.633316 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35654ce-f0b3-46fe-805f-a0f4f7424814 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:49.633588 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35654ce-f0b3-46fe-805f-a0f4f7424814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:49.648820 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-a35654ce-f0b3-46fe-805f-a0f4f7424814 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:10:49.649989 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-a35654ce-f0b3-46fe-805f-a0f4f7424814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:10:49.694249 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 291/596] 10.222.0.58 () {58 vars in 1439 bytes} [Tue May 17 19:10:49 2022] PUT /identity/v3/projects/1a9d3a0631284c9fb113266036993e2d/users/65c1f878b0dc4d63a4f5dcb12fac1dc1/roles/1754a95b778c4e20b53b20cb52ada92b => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:10:49.695024 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e779bfb7-21a0-453a-b658-9a09ab547528 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:49.714682 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 306/597] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:10:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 17 19:10:49.715952 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35654ce-f0b3-46fe-805f-a0f4f7424814 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:49.838189 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35654ce-f0b3-46fe-805f-a0f4f7424814 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:49.838844 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e779bfb7-21a0-453a-b658-9a09ab547528 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:49.875384 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d277937-daf8-4103-860f-23df03cf38de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:49.875757 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d277937-daf8-4103-860f-23df03cf38de None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:49.876004 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d277937-daf8-4103-860f-23df03cf38de None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:49.876164 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032f4f95-a363-4c66-87f0-ec60837449fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:49.876271 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d277937-daf8-4103-860f-23df03cf38de None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:49.876905 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032f4f95-a363-4c66-87f0-ec60837449fa None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:49.877283 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032f4f95-a363-4c66-87f0-ec60837449fa None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:49.877555 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032f4f95-a363-4c66-87f0-ec60837449fa None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:49.878342 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032f4f95-a363-4c66-87f0-ec60837449fa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:49.878693 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032f4f95-a363-4c66-87f0-ec60837449fa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:49.880659 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032f4f95-a363-4c66-87f0-ec60837449fa None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:49.884349 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d277937-daf8-4103-860f-23df03cf38de None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:49.885814 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d277937-daf8-4103-860f-23df03cf38de None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:49.892610 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 292/598] 10.222.0.58 () {56 vars in 1148 bytes} [Tue May 17 19:10:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:49.894900 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032f4f95-a363-4c66-87f0-ec60837449fa None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:49.900788 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-2d277937-daf8-4103-860f-23df03cf38de None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 17 19:10:49.900788 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:49.900788 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:49.900788 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:49.900788 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:49.900788 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:49.900788 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:49.900788 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:49.900788 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:49.900788 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:49.900788 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:49.900788 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:49.900788 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 17 19:10:49.900788 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_user(user_id) May 17 19:10:49.900788 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 17 19:10:49.900788 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 17 19:10:49.901716 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:49.901716 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:49.901716 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 17 19:10:49.901716 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 17 19:10:49.901716 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 17 19:10:49.901716 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 17 19:10:49.901716 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:49.901716 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:49.901716 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:49.901716 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:49.901716 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:49.901716 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:10:49.901716 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:49.901716 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:49.901716 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:49.901716 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:49.901716 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:49.901716 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:49.902643 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:49.902643 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:49.902643 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 17 19:10:49.902643 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 17 19:10:49.902643 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 17 19:10:49.902643 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 17 19:10:49.902643 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 17 19:10:49.902643 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 17 19:10:49.902643 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 17 19:10:49.902643 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:10:49.902643 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 307/599] 10.222.0.58 () {56 vars in 1102 bytes} [Tue May 17 19:10:49 2022] GET /identity/v3/users/neutron => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:10:49.903228 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d277937-daf8-4103-860f-23df03cf38de None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:49.995771 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032f4f95-a363-4c66-87f0-ec60837449fa None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:49.999714 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d277937-daf8-4103-860f-23df03cf38de None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:50.028755 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665df376-4097-4071-b909-c0f4b17423d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:50.029505 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665df376-4097-4071-b909-c0f4b17423d2 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:50.029784 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665df376-4097-4071-b909-c0f4b17423d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:50.030053 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665df376-4097-4071-b909-c0f4b17423d2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:50.033829 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b03a8f-aec3-4a0c-9d21-ec033ccc9b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:50.034853 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b03a8f-aec3-4a0c-9d21-ec033ccc9b37 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:50.035328 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b03a8f-aec3-4a0c-9d21-ec033ccc9b37 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:50.036393 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b03a8f-aec3-4a0c-9d21-ec033ccc9b37 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:50.037225 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665df376-4097-4071-b909-c0f4b17423d2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:50.038763 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665df376-4097-4071-b909-c0f4b17423d2 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:50.046864 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b03a8f-aec3-4a0c-9d21-ec033ccc9b37 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 17 19:10:50.052902 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-665df376-4097-4071-b909-c0f4b17423d2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 17 19:10:50.052902 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:50.052902 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:50.052902 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:50.052902 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:50.052902 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:50.052902 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:50.052902 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:50.052902 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:50.052902 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:50.052902 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:50.052902 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:50.052902 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 17 19:10:50.052902 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(project_id) May 17 19:10:50.052902 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 17 19:10:50.053763 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 17 19:10:50.053763 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:50.053763 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:50.053763 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:50.053763 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:50.053763 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:50.053763 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:50.053763 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:50.053763 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:10:50.053763 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:50.053763 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:50.053763 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:50.053763 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:50.053763 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:50.053763 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:50.053763 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:50.053763 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:50.054648 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 17 19:10:50.054648 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 17 19:10:50.054648 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 17 19:10:50.054648 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 17 19:10:50.054648 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 17 19:10:50.054648 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 17 19:10:50.054648 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 17 19:10:50.054648 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:10:50.054648 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 293/600] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 17 19:10:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:10:50.055127 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665df376-4097-4071-b909-c0f4b17423d2 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:50.057728 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-55b03a8f-aec3-4a0c-9d21-ec033ccc9b37 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 17 19:10:50.057728 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:50.057728 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:50.057728 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:50.057728 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:50.057728 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:50.057728 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:50.057728 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:50.057728 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:50.057728 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:50.057728 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:50.057728 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:50.057728 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 17 19:10:50.057728 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) May 17 19:10:50.057728 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 17 19:10:50.057728 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 17 19:10:50.064949 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 17 19:10:50.064949 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 17 19:10:50.064949 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 17 19:10:50.064949 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 17 19:10:50.064949 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:50.064949 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:50.064949 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:50.064949 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:50.064949 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:50.064949 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:50.064949 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:50.064949 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:10:50.064949 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:50.064949 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:50.064949 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:50.064949 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:50.065829 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:50.065829 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:50.065829 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:50.065829 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:50.065829 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 17 19:10:50.065829 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 17 19:10:50.065829 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 17 19:10:50.065829 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 17 19:10:50.065829 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 17 19:10:50.065829 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 17 19:10:50.065829 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 17 19:10:50.065829 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:10:50.065829 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 308/601] 10.222.0.58 () {56 vars in 1102 bytes} [Tue May 17 19:10:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:10:50.065829 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b03a8f-aec3-4a0c-9d21-ec033ccc9b37 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:50.155369 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b03a8f-aec3-4a0c-9d21-ec033ccc9b37 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:50.157478 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665df376-4097-4071-b909-c0f4b17423d2 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:50.188834 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81033ffd-7814-4db5-ac4f-4276c4fec6cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:50.189775 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81033ffd-7814-4db5-ac4f-4276c4fec6cd None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:50.189973 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f801a0b6-2621-4264-9ca2-09cfa701ac5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:50.190100 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81033ffd-7814-4db5-ac4f-4276c4fec6cd None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:50.190310 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81033ffd-7814-4db5-ac4f-4276c4fec6cd None admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:50.190844 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f801a0b6-2621-4264-9ca2-09cfa701ac5c None admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:50.191121 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f801a0b6-2621-4264-9ca2-09cfa701ac5c None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:50.191216 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81033ffd-7814-4db5-ac4f-4276c4fec6cd None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:50.191395 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f801a0b6-2621-4264-9ca2-09cfa701ac5c None admin] PATH_INFO: `/v3/projects/1a9d3a0631284c9fb113266036993e2d/users/57e888884d2b4629bfd12d1034ce5be4/roles/1754a95b778c4e20b53b20cb52ada92b` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:50.191582 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81033ffd-7814-4db5-ac4f-4276c4fec6cd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:50.192932 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81033ffd-7814-4db5-ac4f-4276c4fec6cd None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:50.205755 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f801a0b6-2621-4264-9ca2-09cfa701ac5c None admin] RBAC: Authorizing `identity:create_grant(project_id=1a9d3a0631284c9fb113266036993e2d, user_id=57e888884d2b4629bfd12d1034ce5be4, role_id=1754a95b778c4e20b53b20cb52ada92b)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:50.207130 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f801a0b6-2621-4264-9ca2-09cfa701ac5c None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:50.211139 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 309/602] 10.222.0.58 () {56 vars in 1139 bytes} [Tue May 17 19:10:50 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:50.212154 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81033ffd-7814-4db5-ac4f-4276c4fec6cd None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:50.303390 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 294/603] 10.222.0.58 () {58 vars in 1439 bytes} [Tue May 17 19:10:50 2022] PUT /identity/v3/projects/1a9d3a0631284c9fb113266036993e2d/users/57e888884d2b4629bfd12d1034ce5be4/roles/1754a95b778c4e20b53b20cb52ada92b => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:10:50.304589 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f801a0b6-2621-4264-9ca2-09cfa701ac5c None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:50.306661 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81033ffd-7814-4db5-ac4f-4276c4fec6cd None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:50.349365 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bacd85-d3fc-4931-8da9-9ba15f85cbcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:50.350608 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bacd85-d3fc-4931-8da9-9ba15f85cbcc None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:50.350953 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bacd85-d3fc-4931-8da9-9ba15f85cbcc None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:50.351233 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bacd85-d3fc-4931-8da9-9ba15f85cbcc None admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:50.352434 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bacd85-d3fc-4931-8da9-9ba15f85cbcc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:50.352747 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bacd85-d3fc-4931-8da9-9ba15f85cbcc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:50.354459 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bacd85-d3fc-4931-8da9-9ba15f85cbcc None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:50.370843 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 310/604] 10.222.0.58 () {56 vars in 1148 bytes} [Tue May 17 19:10:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:50.371828 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bacd85-d3fc-4931-8da9-9ba15f85cbcc None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:50.447773 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f801a0b6-2621-4264-9ca2-09cfa701ac5c None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:50.493261 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6b9594-2646-4605-950c-1fcd22d1491f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:50.493985 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6b9594-2646-4605-950c-1fcd22d1491f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:50.494330 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6b9594-2646-4605-950c-1fcd22d1491f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:50.494670 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6b9594-2646-4605-950c-1fcd22d1491f None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:50.495509 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6b9594-2646-4605-950c-1fcd22d1491f None admin] RBAC: Adding query filter params (name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:50.495884 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6b9594-2646-4605-950c-1fcd22d1491f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:50.497332 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6b9594-2646-4605-950c-1fcd22d1491f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:50.504819 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bacd85-d3fc-4931-8da9-9ba15f85cbcc None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:50.506962 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 295/605] 10.222.0.58 () {56 vars in 1103 bytes} [Tue May 17 19:10:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:50.508254 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6b9594-2646-4605-950c-1fcd22d1491f None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:50.548172 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0544b56-abab-4594-8bfe-c2f2d6f0927a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:50.549298 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0544b56-abab-4594-8bfe-c2f2d6f0927a None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:50.549771 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0544b56-abab-4594-8bfe-c2f2d6f0927a None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:50.550220 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0544b56-abab-4594-8bfe-c2f2d6f0927a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:50.562653 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0544b56-abab-4594-8bfe-c2f2d6f0927a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:50.564372 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0544b56-abab-4594-8bfe-c2f2d6f0927a None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:50.581548 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-b0544b56-abab-4594-8bfe-c2f2d6f0927a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:10:50.581548 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:50.581548 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:50.581548 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:50.581548 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:50.581548 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:50.581548 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:50.581548 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:50.581548 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:50.581548 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:50.581548 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:50.581548 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:50.581548 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 17 19:10:50.581548 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 17 19:10:50.581548 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 17 19:10:50.582504 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 17 19:10:50.582504 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:50.582504 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:50.582504 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:50.582504 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:50.582504 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:50.582504 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:50.582504 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:50.582504 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:10:50.582504 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:50.582504 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:50.582504 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:50.582504 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:50.582504 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:50.582504 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:50.582504 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:50.582504 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:50.583399 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 17 19:10:50.583399 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 17 19:10:50.583399 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:10:50.583399 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:10:50.589526 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 311/606] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:10:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:10:50.590564 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0544b56-abab-4594-8bfe-c2f2d6f0927a None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:50.609396 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6b9594-2646-4605-950c-1fcd22d1491f None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:50.643208 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e477134f-8e1e-4356-a4c4-eb1ef33eda38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:50.644315 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e477134f-8e1e-4356-a4c4-eb1ef33eda38 None admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:50.644440 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e477134f-8e1e-4356-a4c4-eb1ef33eda38 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:50.644654 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e477134f-8e1e-4356-a4c4-eb1ef33eda38 None admin] PATH_INFO: `/v3/projects/1a9d3a0631284c9fb113266036993e2d/users/d05409b38ae346458abfe41ebda1c59e/roles/1754a95b778c4e20b53b20cb52ada92b` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:50.659182 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e477134f-8e1e-4356-a4c4-eb1ef33eda38 None admin] RBAC: Authorizing `identity:create_grant(project_id=1a9d3a0631284c9fb113266036993e2d, user_id=d05409b38ae346458abfe41ebda1c59e, role_id=1754a95b778c4e20b53b20cb52ada92b)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:50.661228 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e477134f-8e1e-4356-a4c4-eb1ef33eda38 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:50.692677 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0544b56-abab-4594-8bfe-c2f2d6f0927a None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:50.726159 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49b7a5d-621e-487a-bf11-1e6bd242b446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:50.726949 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49b7a5d-621e-487a-bf11-1e6bd242b446 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:50.727219 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49b7a5d-621e-487a-bf11-1e6bd242b446 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:50.727528 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49b7a5d-621e-487a-bf11-1e6bd242b446 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:50.737626 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 296/607] 10.222.0.58 () {58 vars in 1439 bytes} [Tue May 17 19:10:50 2022] PUT /identity/v3/projects/1a9d3a0631284c9fb113266036993e2d/users/d05409b38ae346458abfe41ebda1c59e/roles/1754a95b778c4e20b53b20cb52ada92b => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:10:50.738146 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49b7a5d-621e-487a-bf11-1e6bd242b446 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:50.739197 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e477134f-8e1e-4356-a4c4-eb1ef33eda38 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:50.739666 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49b7a5d-621e-487a-bf11-1e6bd242b446 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:50.758644 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-f49b7a5d-621e-487a-bf11-1e6bd242b446 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:10:50.758644 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:50.758644 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:50.758644 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:50.758644 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:50.758644 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:50.758644 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:50.758644 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:50.758644 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:50.758644 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:50.758644 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:50.758644 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:50.758644 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 17 19:10:50.758644 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 17 19:10:50.758644 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 17 19:10:50.759943 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 17 19:10:50.759943 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:50.759943 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:50.759943 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:50.759943 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:50.759943 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:50.759943 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:50.759943 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:50.759943 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:10:50.759943 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:50.759943 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:50.759943 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:50.759943 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:50.759943 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:50.759943 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:50.759943 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:50.759943 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:50.761287 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 17 19:10:50.761287 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 17 19:10:50.761287 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:10:50.761287 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:10:50.761287 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 312/608] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:10:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:10:50.765526 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49b7a5d-621e-487a-bf11-1e6bd242b446 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:50.852655 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e477134f-8e1e-4356-a4c4-eb1ef33eda38 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:50.870335 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49b7a5d-621e-487a-bf11-1e6bd242b446 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:50.893801 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51005b1b-664a-4978-b411-59ba96b5f6d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:50.895441 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51005b1b-664a-4978-b411-59ba96b5f6d9 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:50.895979 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51005b1b-664a-4978-b411-59ba96b5f6d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:50.896481 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51005b1b-664a-4978-b411-59ba96b5f6d9 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:50.897530 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51005b1b-664a-4978-b411-59ba96b5f6d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:50.898114 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51005b1b-664a-4978-b411-59ba96b5f6d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:50.899763 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51005b1b-664a-4978-b411-59ba96b5f6d9 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:50.907019 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff94afb7-ac64-4a8a-8dfd-740f2358c75e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:50.907729 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff94afb7-ac64-4a8a-8dfd-740f2358c75e None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:50.908037 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff94afb7-ac64-4a8a-8dfd-740f2358c75e None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:50.908852 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff94afb7-ac64-4a8a-8dfd-740f2358c75e None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:50.909037 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff94afb7-ac64-4a8a-8dfd-740f2358c75e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:50.909316 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff94afb7-ac64-4a8a-8dfd-740f2358c75e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:50.910800 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff94afb7-ac64-4a8a-8dfd-740f2358c75e None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:50.919640 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 297/609] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 17 19:10:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:50.921248 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51005b1b-664a-4978-b411-59ba96b5f6d9 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:50.924192 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 313/610] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 17 19:10:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:10:50.930211 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff94afb7-ac64-4a8a-8dfd-740f2358c75e None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:51.028020 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51005b1b-664a-4978-b411-59ba96b5f6d9 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:51.034919 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff94afb7-ac64-4a8a-8dfd-740f2358c75e None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:51.064033 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30875a13-7ce6-4654-ab0b-13c69a8bc7d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:51.065230 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30875a13-7ce6-4654-ab0b-13c69a8bc7d3 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:51.065705 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30875a13-7ce6-4654-ab0b-13c69a8bc7d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:51.066168 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30875a13-7ce6-4654-ab0b-13c69a8bc7d3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:51.070804 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac4982a-7ca1-4a28-9784-6e94fa6a57e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:51.071624 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac4982a-7ca1-4a28-9784-6e94fa6a57e0 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:51.071907 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac4982a-7ca1-4a28-9784-6e94fa6a57e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:51.072180 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac4982a-7ca1-4a28-9784-6e94fa6a57e0 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:51.075313 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30875a13-7ce6-4654-ab0b-13c69a8bc7d3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:51.077098 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30875a13-7ce6-4654-ab0b-13c69a8bc7d3 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:51.080245 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac4982a-7ca1-4a28-9784-6e94fa6a57e0 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:51.081708 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac4982a-7ca1-4a28-9784-6e94fa6a57e0 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:51.093609 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-30875a13-7ce6-4654-ab0b-13c69a8bc7d3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 17 19:10:51.093609 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:51.093609 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:51.093609 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:51.093609 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:51.093609 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:51.093609 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:51.093609 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:51.093609 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:51.093609 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:51.093609 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:51.093609 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:51.093609 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 17 19:10:51.093609 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(project_id) May 17 19:10:51.093609 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 17 19:10:51.094559 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 17 19:10:51.094559 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:51.094559 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:51.094559 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:51.094559 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:51.094559 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:51.094559 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:51.094559 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:51.094559 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:10:51.094559 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:51.094559 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:51.094559 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:51.094559 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:51.094559 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:51.094559 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:51.094559 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:51.094559 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:51.095471 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 17 19:10:51.095471 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 17 19:10:51.095471 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 17 19:10:51.095471 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 17 19:10:51.095471 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 17 19:10:51.095471 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 17 19:10:51.095471 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 17 19:10:51.095471 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:10:51.097645 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-fac4982a-7ca1-4a28-9784-6e94fa6a57e0 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 17 19:10:51.097645 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:51.097645 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:51.097645 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:51.097645 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:51.097645 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:51.097645 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:51.097645 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:51.097645 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:51.097645 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:51.097645 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:51.097645 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:51.097645 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 17 19:10:51.097645 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_user(user_id) May 17 19:10:51.097645 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 17 19:10:51.097645 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 17 19:10:51.098651 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:51.098651 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:51.098651 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 17 19:10:51.098651 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 17 19:10:51.098651 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 17 19:10:51.098651 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 17 19:10:51.098651 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:51.098651 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:51.098651 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:51.098651 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:51.098651 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:51.098651 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:10:51.098651 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:51.098651 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:51.098651 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:51.098651 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:51.098651 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:51.098651 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:51.099675 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:51.099675 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:51.099675 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 17 19:10:51.099675 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 17 19:10:51.099675 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 17 19:10:51.099675 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 17 19:10:51.099675 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 17 19:10:51.099675 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 17 19:10:51.099675 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 17 19:10:51.099675 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:10:51.100831 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 314/611] 10.222.0.58 () {56 vars in 1096 bytes} [Tue May 17 19:10:50 2022] GET /identity/v3/users/swift => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:10:51.104084 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 298/612] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 17 19:10:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 17 19:10:51.105307 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac4982a-7ca1-4a28-9784-6e94fa6a57e0 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:51.107574 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30875a13-7ce6-4654-ab0b-13c69a8bc7d3 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:51.207753 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac4982a-7ca1-4a28-9784-6e94fa6a57e0 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:51.208376 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30875a13-7ce6-4654-ab0b-13c69a8bc7d3 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:51.242586 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d357e39-01a3-47a9-b7f2-aba5913c39cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:51.243256 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d357e39-01a3-47a9-b7f2-aba5913c39cc None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:51.243556 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d357e39-01a3-47a9-b7f2-aba5913c39cc None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:51.243834 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d357e39-01a3-47a9-b7f2-aba5913c39cc None admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:51.244770 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d357e39-01a3-47a9-b7f2-aba5913c39cc None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:51.245079 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d357e39-01a3-47a9-b7f2-aba5913c39cc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:51.245673 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fd2c9d-f249-435e-b7f3-c3f20abdf0dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:51.246502 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d357e39-01a3-47a9-b7f2-aba5913c39cc None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:51.246723 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fd2c9d-f249-435e-b7f3-c3f20abdf0dc None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:51.247198 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fd2c9d-f249-435e-b7f3-c3f20abdf0dc None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:51.247714 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fd2c9d-f249-435e-b7f3-c3f20abdf0dc None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:51.248697 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fd2c9d-f249-435e-b7f3-c3f20abdf0dc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:51.249192 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fd2c9d-f249-435e-b7f3-c3f20abdf0dc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:51.250926 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fd2c9d-f249-435e-b7f3-c3f20abdf0dc None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:51.265972 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 315/614] 10.222.0.58 () {56 vars in 1135 bytes} [Tue May 17 19:10:51 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:51.266334 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 299/614] 10.222.0.58 () {56 vars in 1148 bytes} [Tue May 17 19:10:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:10:51.270068 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d357e39-01a3-47a9-b7f2-aba5913c39cc None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:51.277077 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fd2c9d-f249-435e-b7f3-c3f20abdf0dc None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:51.367887 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d357e39-01a3-47a9-b7f2-aba5913c39cc None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:51.384355 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fd2c9d-f249-435e-b7f3-c3f20abdf0dc None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:51.401614 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fb2cfe-b10d-4ba1-85a8-737e741caeb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:51.402461 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fb2cfe-b10d-4ba1-85a8-737e741caeb7 None admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:51.402734 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fb2cfe-b10d-4ba1-85a8-737e741caeb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:51.403003 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fb2cfe-b10d-4ba1-85a8-737e741caeb7 None admin] PATH_INFO: `/v3/projects/1a9d3a0631284c9fb113266036993e2d/users/1ac193d15f48463c98ff1d97770ed794/roles/1754a95b778c4e20b53b20cb52ada92b` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:51.418042 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fb2cfe-b10d-4ba1-85a8-737e741caeb7 None admin] RBAC: Authorizing `identity:create_grant(project_id=1a9d3a0631284c9fb113266036993e2d, user_id=1ac193d15f48463c98ff1d97770ed794, role_id=1754a95b778c4e20b53b20cb52ada92b)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:51.419513 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fb2cfe-b10d-4ba1-85a8-737e741caeb7 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:51.420343 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5b65c2-1c6c-46b2-ab38-e68b8cea1ae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:51.421364 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5b65c2-1c6c-46b2-ab38-e68b8cea1ae4 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:51.421837 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5b65c2-1c6c-46b2-ab38-e68b8cea1ae4 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:51.422318 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5b65c2-1c6c-46b2-ab38-e68b8cea1ae4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:51.434452 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5b65c2-1c6c-46b2-ab38-e68b8cea1ae4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:51.436320 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5b65c2-1c6c-46b2-ab38-e68b8cea1ae4 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:51.454395 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-bb5b65c2-1c6c-46b2-ab38-e68b8cea1ae4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:10:51.454395 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:51.454395 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:51.454395 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:51.454395 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:51.454395 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:51.454395 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:51.454395 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:51.454395 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:51.454395 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:51.454395 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:51.454395 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:51.454395 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 17 19:10:51.454395 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 17 19:10:51.454395 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 17 19:10:51.455289 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 17 19:10:51.455289 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:51.455289 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:51.455289 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:51.455289 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:51.455289 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:51.455289 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:51.455289 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:51.455289 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:10:51.455289 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:51.455289 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:51.455289 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:51.455289 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:51.455289 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:51.455289 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:51.455289 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:51.455289 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:51.456217 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 17 19:10:51.456217 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 17 19:10:51.456217 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:10:51.456217 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:10:51.456217 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 300/615] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:10:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 17 19:10:51.464449 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5b65c2-1c6c-46b2-ab38-e68b8cea1ae4 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:51.495271 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 316/616] 10.222.0.58 () {58 vars in 1439 bytes} [Tue May 17 19:10:51 2022] PUT /identity/v3/projects/1a9d3a0631284c9fb113266036993e2d/users/1ac193d15f48463c98ff1d97770ed794/roles/1754a95b778c4e20b53b20cb52ada92b => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:10:51.590980 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5b65c2-1c6c-46b2-ab38-e68b8cea1ae4 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:51.635245 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4d8d0d-94f8-4f9e-83c3-204b96397f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:51.635927 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4d8d0d-94f8-4f9e-83c3-204b96397f5f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:51.636148 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4d8d0d-94f8-4f9e-83c3-204b96397f5f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:51.636424 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4d8d0d-94f8-4f9e-83c3-204b96397f5f None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:51.637151 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4d8d0d-94f8-4f9e-83c3-204b96397f5f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:51.637454 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4d8d0d-94f8-4f9e-83c3-204b96397f5f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:51.638836 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4d8d0d-94f8-4f9e-83c3-204b96397f5f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:51.656749 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 301/617] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 17 19:10:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:51.662406 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fb2cfe-b10d-4ba1-85a8-737e741caeb7 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:51.781078 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fb2cfe-b10d-4ba1-85a8-737e741caeb7 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:51.832927 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4169466-611b-4127-8ba1-e48e1b41f922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:51.833764 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4169466-611b-4127-8ba1-e48e1b41f922 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:51.834042 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4169466-611b-4127-8ba1-e48e1b41f922 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:51.834312 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4169466-611b-4127-8ba1-e48e1b41f922 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:51.842162 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4169466-611b-4127-8ba1-e48e1b41f922 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:51.843701 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4169466-611b-4127-8ba1-e48e1b41f922 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:51.864778 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-a4169466-611b-4127-8ba1-e48e1b41f922 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 17 19:10:51.864778 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:51.864778 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:51.864778 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:51.864778 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:51.864778 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:51.864778 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:51.864778 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:51.864778 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:51.864778 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:51.864778 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:51.864778 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:51.864778 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 17 19:10:51.864778 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(project_id) May 17 19:10:51.864778 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 17 19:10:51.865732 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 17 19:10:51.865732 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:51.865732 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:51.865732 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:51.865732 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:51.865732 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:51.865732 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:51.865732 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:51.865732 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:10:51.865732 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:51.865732 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:51.865732 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:51.865732 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:51.865732 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:51.865732 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:51.865732 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:51.865732 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:51.866628 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 17 19:10:51.866628 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 17 19:10:51.866628 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 17 19:10:51.866628 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 17 19:10:51.866628 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 17 19:10:51.866628 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 17 19:10:51.866628 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 17 19:10:51.866628 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:10:51.866628 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 317/618] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 17 19:10:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:10:51.871917 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4d8d0d-94f8-4f9e-83c3-204b96397f5f None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:51.995747 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4d8d0d-94f8-4f9e-83c3-204b96397f5f None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:52.039826 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c36cd2d-b109-4687-aa05-91acdbe2e24f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:52.040599 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c36cd2d-b109-4687-aa05-91acdbe2e24f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:52.040774 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c36cd2d-b109-4687-aa05-91acdbe2e24f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:52.041036 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c36cd2d-b109-4687-aa05-91acdbe2e24f None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:52.041896 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c36cd2d-b109-4687-aa05-91acdbe2e24f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:52.042250 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c36cd2d-b109-4687-aa05-91acdbe2e24f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:52.043839 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c36cd2d-b109-4687-aa05-91acdbe2e24f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:52.062911 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4169466-611b-4127-8ba1-e48e1b41f922 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:52.063717 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 302/619] 10.222.0.58 () {56 vars in 1148 bytes} [Tue May 17 19:10:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:52.223959 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4169466-611b-4127-8ba1-e48e1b41f922 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:52.237756 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc532da-e46a-405c-bac0-953bc5c3b98b None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:52.238348 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc532da-e46a-405c-bac0-953bc5c3b98b None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:52.239405 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc532da-e46a-405c-bac0-953bc5c3b98b None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:52.241216 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 303/620] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:10:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 17 19:10:52.247516 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11717fcb-0d52-4b3c-93d2-781ac0375ee5 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:52.248049 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11717fcb-0d52-4b3c-93d2-781ac0375ee5 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:52.248648 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11717fcb-0d52-4b3c-93d2-781ac0375ee5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:52.267861 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-11717fcb-0d52-4b3c-93d2-781ac0375ee5 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:10:52.269791 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-11717fcb-0d52-4b3c-93d2-781ac0375ee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:10:52.280545 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42671013-592e-4e56-8c4e-a081c9b1c990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:52.281730 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42671013-592e-4e56-8c4e-a081c9b1c990 None admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:52.283333 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42671013-592e-4e56-8c4e-a081c9b1c990 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:52.283824 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42671013-592e-4e56-8c4e-a081c9b1c990 None admin] PATH_INFO: `/v3/projects/1a9d3a0631284c9fb113266036993e2d/users/b0cb292002434d30bc81f370d120a2a2/roles/1754a95b778c4e20b53b20cb52ada92b` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:52.311934 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42671013-592e-4e56-8c4e-a081c9b1c990 None admin] RBAC: Authorizing `identity:create_grant(project_id=1a9d3a0631284c9fb113266036993e2d, user_id=b0cb292002434d30bc81f370d120a2a2, role_id=1754a95b778c4e20b53b20cb52ada92b)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:52.314678 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42671013-592e-4e56-8c4e-a081c9b1c990 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:52.360443 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 304/621] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:10:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 17 19:10:52.384850 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11717fcb-0d52-4b3c-93d2-781ac0375ee5 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:52.433848 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 318/622] 10.222.0.58 () {58 vars in 1439 bytes} [Tue May 17 19:10:52 2022] PUT /identity/v3/projects/1a9d3a0631284c9fb113266036993e2d/users/b0cb292002434d30bc81f370d120a2a2/roles/1754a95b778c4e20b53b20cb52ada92b => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:10:52.493689 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11717fcb-0d52-4b3c-93d2-781ac0375ee5 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:52.535152 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3dc1ca-2e05-44e5-867c-021f5e9f0524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:52.536038 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3dc1ca-2e05-44e5-867c-021f5e9f0524 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:52.536308 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3dc1ca-2e05-44e5-867c-021f5e9f0524 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:52.536579 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3dc1ca-2e05-44e5-867c-021f5e9f0524 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:52.547721 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3dc1ca-2e05-44e5-867c-021f5e9f0524 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 17 19:10:52.556181 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-4d3dc1ca-2e05-44e5-867c-021f5e9f0524 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 17 19:10:52.556181 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:52.556181 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:52.556181 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:52.556181 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:52.556181 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:52.556181 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:52.556181 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:52.556181 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:52.556181 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:52.556181 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:52.556181 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:52.556181 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 17 19:10:52.556181 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) May 17 19:10:52.556181 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 17 19:10:52.556181 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 17 19:10:52.557169 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 17 19:10:52.557169 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 17 19:10:52.557169 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 17 19:10:52.557169 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 17 19:10:52.557169 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:52.557169 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:52.557169 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:52.557169 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:52.557169 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:52.557169 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:52.557169 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:52.557169 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:10:52.557169 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:52.557169 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:52.557169 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:52.557169 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:52.557989 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:52.557989 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:52.557989 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:52.557989 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:52.557989 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 17 19:10:52.557989 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 17 19:10:52.557989 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 17 19:10:52.557989 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 17 19:10:52.557989 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 17 19:10:52.557989 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 17 19:10:52.557989 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 17 19:10:52.557989 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:10:52.557989 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 305/623] 10.222.0.58 () {56 vars in 1102 bytes} [Tue May 17 19:10:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 17 19:10:52.565039 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42671013-592e-4e56-8c4e-a081c9b1c990 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:52.695136 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42671013-592e-4e56-8c4e-a081c9b1c990 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:52.731764 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890bccaf-f2e1-4007-b095-0091bd65d5c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:52.732889 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890bccaf-f2e1-4007-b095-0091bd65d5c5 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:52.733393 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890bccaf-f2e1-4007-b095-0091bd65d5c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:52.733861 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890bccaf-f2e1-4007-b095-0091bd65d5c5 None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:52.734830 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890bccaf-f2e1-4007-b095-0091bd65d5c5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:52.735333 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890bccaf-f2e1-4007-b095-0091bd65d5c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:52.736980 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890bccaf-f2e1-4007-b095-0091bd65d5c5 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:52.747369 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 319/624] 10.222.0.58 () {56 vars in 1103 bytes} [Tue May 17 19:10:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:52.753505 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3dc1ca-2e05-44e5-867c-021f5e9f0524 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:52.882475 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3dc1ca-2e05-44e5-867c-021f5e9f0524 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:52.922952 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffa4411-e3bf-4924-b3ee-88c3147b03b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:52.923782 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffa4411-e3bf-4924-b3ee-88c3147b03b8 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:52.924066 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffa4411-e3bf-4924-b3ee-88c3147b03b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:52.924343 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffa4411-e3bf-4924-b3ee-88c3147b03b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:52.935467 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffa4411-e3bf-4924-b3ee-88c3147b03b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:52.936932 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffa4411-e3bf-4924-b3ee-88c3147b03b8 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:52.958426 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-fffa4411-e3bf-4924-b3ee-88c3147b03b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:10:52.958426 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:52.958426 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:52.958426 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:52.958426 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:52.958426 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:52.958426 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:52.958426 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:52.958426 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:52.958426 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:52.958426 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:52.958426 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:52.958426 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 17 19:10:52.958426 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 17 19:10:52.958426 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 17 19:10:52.959290 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 17 19:10:52.959290 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:52.959290 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:52.959290 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:52.959290 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:52.959290 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:52.959290 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:52.959290 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:52.959290 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:10:52.959290 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:52.959290 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:52.959290 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:52.959290 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:52.959290 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:52.959290 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:52.959290 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:52.959290 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:52.960261 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 17 19:10:52.960261 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 17 19:10:52.960261 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:10:52.960261 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:10:52.963998 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 306/625] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:10:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:10:52.969352 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890bccaf-f2e1-4007-b095-0091bd65d5c5 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:52.988409 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2f245b-e525-4ff9-80af-7eba2f642aea None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:52.988793 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2f245b-e525-4ff9-80af-7eba2f642aea None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:52.989075 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2f245b-e525-4ff9-80af-7eba2f642aea None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:52.990558 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 307/626] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:10:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 17 19:10:52.997905 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4f8c79-5bb4-48da-bf98-619cce629613 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:52.998117 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4f8c79-5bb4-48da-bf98-619cce629613 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:52.998395 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4f8c79-5bb4-48da-bf98-619cce629613 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:53.015473 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-fa4f8c79-5bb4-48da-bf98-619cce629613 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:10:53.016752 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-fa4f8c79-5bb4-48da-bf98-619cce629613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:10:53.084970 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 308/627] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:10:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 17 19:10:53.119106 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4f8c79-5bb4-48da-bf98-619cce629613 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:53.179989 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890bccaf-f2e1-4007-b095-0091bd65d5c5 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:53.228025 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4f8c79-5bb4-48da-bf98-619cce629613 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:53.266350 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f41250f-334b-4fa1-b8d9-6a3660e84018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:53.267073 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f41250f-334b-4fa1-b8d9-6a3660e84018 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:53.267442 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f41250f-334b-4fa1-b8d9-6a3660e84018 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:53.267752 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f41250f-334b-4fa1-b8d9-6a3660e84018 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:53.270310 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bb6f8b-7a4f-43c1-9e7c-f3ba3eadc360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:53.271187 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bb6f8b-7a4f-43c1-9e7c-f3ba3eadc360 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:53.271698 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bb6f8b-7a4f-43c1-9e7c-f3ba3eadc360 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:53.272181 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bb6f8b-7a4f-43c1-9e7c-f3ba3eadc360 None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:53.273100 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bb6f8b-7a4f-43c1-9e7c-f3ba3eadc360 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:53.273593 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bb6f8b-7a4f-43c1-9e7c-f3ba3eadc360 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:53.275132 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bb6f8b-7a4f-43c1-9e7c-f3ba3eadc360 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:53.280505 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f41250f-334b-4fa1-b8d9-6a3660e84018 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 17 19:10:53.286549 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-9f41250f-334b-4fa1-b8d9-6a3660e84018 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 17 19:10:53.286549 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:53.286549 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:53.286549 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:53.286549 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:53.286549 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:53.286549 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:53.286549 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:53.286549 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:53.286549 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:53.286549 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:53.286549 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:53.286549 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 17 19:10:53.286549 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) May 17 19:10:53.286549 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 17 19:10:53.286549 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 17 19:10:53.287471 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 17 19:10:53.287471 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 17 19:10:53.287471 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 17 19:10:53.287471 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 17 19:10:53.287471 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:53.287471 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:53.287471 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:53.287471 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:53.287471 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:53.287471 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:53.287471 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:53.287471 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:10:53.287471 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:53.287471 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:53.287471 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:53.287471 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:53.288516 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:53.288516 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:53.288516 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:53.288516 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:53.288516 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 17 19:10:53.288516 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 17 19:10:53.288516 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 17 19:10:53.288516 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 17 19:10:53.288516 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 17 19:10:53.288516 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 17 19:10:53.288516 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 17 19:10:53.288516 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:10:53.288516 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 309/628] 10.222.0.58 () {56 vars in 1102 bytes} [Tue May 17 19:10:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 17 19:10:53.294361 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f41250f-334b-4fa1-b8d9-6a3660e84018 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:53.299847 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 320/629] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 17 19:10:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:53.305967 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bb6f8b-7a4f-43c1-9e7c-f3ba3eadc360 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:53.445647 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f41250f-334b-4fa1-b8d9-6a3660e84018 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:53.482101 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3edd8f-3240-4502-bde8-de6d239e9519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:53.482765 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3edd8f-3240-4502-bde8-de6d239e9519 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:53.483042 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3edd8f-3240-4502-bde8-de6d239e9519 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:53.483312 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3edd8f-3240-4502-bde8-de6d239e9519 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:53.484069 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3edd8f-3240-4502-bde8-de6d239e9519 None admin] RBAC: Adding query filter params (name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:53.484665 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3edd8f-3240-4502-bde8-de6d239e9519 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:53.486054 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3edd8f-3240-4502-bde8-de6d239e9519 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:53.500047 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 310/630] 10.222.0.58 () {56 vars in 1103 bytes} [Tue May 17 19:10:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:10:53.501466 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3edd8f-3240-4502-bde8-de6d239e9519 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:53.521717 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bb6f8b-7a4f-43c1-9e7c-f3ba3eadc360 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:53.574288 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dae63f-8af9-428b-b056-06020f0b636b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:53.575083 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dae63f-8af9-428b-b056-06020f0b636b None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:53.575358 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dae63f-8af9-428b-b056-06020f0b636b None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:53.575649 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dae63f-8af9-428b-b056-06020f0b636b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:53.588840 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dae63f-8af9-428b-b056-06020f0b636b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:53.590289 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dae63f-8af9-428b-b056-06020f0b636b None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:53.606930 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-18dae63f-8af9-428b-b056-06020f0b636b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 17 19:10:53.606930 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:53.606930 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:53.606930 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:53.606930 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:53.606930 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:53.606930 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:53.606930 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:53.606930 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:53.606930 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:53.606930 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:53.606930 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:53.606930 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 17 19:10:53.606930 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_user(user_id) May 17 19:10:53.606930 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 17 19:10:53.606930 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 17 19:10:53.607994 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:53.607994 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:53.607994 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 17 19:10:53.607994 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 17 19:10:53.607994 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 17 19:10:53.607994 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 17 19:10:53.607994 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:53.607994 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:53.607994 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:53.607994 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:53.607994 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:53.607994 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:10:53.607994 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:53.607994 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:53.607994 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:53.607994 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:53.607994 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:53.607994 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:53.608943 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:53.608943 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:53.608943 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 17 19:10:53.608943 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 17 19:10:53.608943 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 17 19:10:53.608943 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 17 19:10:53.608943 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 17 19:10:53.608943 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 17 19:10:53.608943 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 17 19:10:53.608943 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:10:53.608943 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 321/631] 10.222.0.58 () {56 vars in 1093 bytes} [Tue May 17 19:10:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:10:53.610024 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5233cdf5-893c-43ba-9d7c-bc4f727fe397 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:53.610024 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5233cdf5-893c-43ba-9d7c-bc4f727fe397 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:53.610325 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5233cdf5-893c-43ba-9d7c-bc4f727fe397 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:53.615809 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 322/632] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:10:53 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:10:53.616595 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5233cdf5-893c-43ba-9d7c-bc4f727fe397 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:53.644860 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3edd8f-3240-4502-bde8-de6d239e9519 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:53.687947 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21435943-b578-4bcd-b0cd-1b7b8c2206ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:53.688745 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21435943-b578-4bcd-b0cd-1b7b8c2206ac None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:53.689023 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21435943-b578-4bcd-b0cd-1b7b8c2206ac None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:53.689289 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21435943-b578-4bcd-b0cd-1b7b8c2206ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:53.702986 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21435943-b578-4bcd-b0cd-1b7b8c2206ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:53.704619 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21435943-b578-4bcd-b0cd-1b7b8c2206ac None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:53.721673 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5233cdf5-893c-43ba-9d7c-bc4f727fe397 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:53.724698 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-21435943-b578-4bcd-b0cd-1b7b8c2206ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:10:53.724698 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:53.724698 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:53.724698 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:53.724698 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:53.724698 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:53.724698 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:53.724698 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:53.724698 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:53.724698 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:53.724698 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:53.724698 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:53.724698 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 17 19:10:53.724698 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 17 19:10:53.724698 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 17 19:10:53.726420 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 17 19:10:53.726420 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:53.726420 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:53.726420 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:53.726420 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:53.726420 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:53.726420 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:53.726420 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:53.726420 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:10:53.726420 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:53.726420 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:53.726420 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:53.726420 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:53.726420 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:53.726420 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:53.726420 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:53.726420 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:53.728667 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 17 19:10:53.728667 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 17 19:10:53.728667 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:10:53.728667 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:10:53.728667 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 311/633] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:10:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:10:53.728667 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28c4de1-32d5-4f09-a164-e95b38fcbba4 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:53.728667 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28c4de1-32d5-4f09-a164-e95b38fcbba4 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:53.729552 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28c4de1-32d5-4f09-a164-e95b38fcbba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:53.744997 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-d28c4de1-32d5-4f09-a164-e95b38fcbba4 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:10:53.746188 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d28c4de1-32d5-4f09-a164-e95b38fcbba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:10:53.762084 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617d718e-2f4c-4af1-8b58-ed916870f0f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:53.762951 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617d718e-2f4c-4af1-8b58-ed916870f0f2 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:53.763265 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617d718e-2f4c-4af1-8b58-ed916870f0f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:53.763852 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617d718e-2f4c-4af1-8b58-ed916870f0f2 None admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:53.764750 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617d718e-2f4c-4af1-8b58-ed916870f0f2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:53.765054 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617d718e-2f4c-4af1-8b58-ed916870f0f2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:53.766940 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617d718e-2f4c-4af1-8b58-ed916870f0f2 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:53.790082 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 323/634] 10.222.0.58 () {56 vars in 1133 bytes} [Tue May 17 19:10:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:53.791223 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617d718e-2f4c-4af1-8b58-ed916870f0f2 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:53.819595 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 312/635] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:10:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 17 19:10:53.820495 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28c4de1-32d5-4f09-a164-e95b38fcbba4 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:53.897890 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617d718e-2f4c-4af1-8b58-ed916870f0f2 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:53.932203 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791f335c-1b12-4af4-a3c2-36dda2dd3aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:53.933023 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791f335c-1b12-4af4-a3c2-36dda2dd3aab None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:53.933296 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791f335c-1b12-4af4-a3c2-36dda2dd3aab None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:53.933766 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791f335c-1b12-4af4-a3c2-36dda2dd3aab None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:53.934805 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791f335c-1b12-4af4-a3c2-36dda2dd3aab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:53.935107 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791f335c-1b12-4af4-a3c2-36dda2dd3aab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:53.936926 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791f335c-1b12-4af4-a3c2-36dda2dd3aab None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:53.940021 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28c4de1-32d5-4f09-a164-e95b38fcbba4 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:53.951266 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 324/636] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 17 19:10:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:53.952335 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791f335c-1b12-4af4-a3c2-36dda2dd3aab None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:53.982965 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a6b21c-f782-44a8-88de-b098468e8b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:53.983979 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a6b21c-f782-44a8-88de-b098468e8b56 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:53.984444 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a6b21c-f782-44a8-88de-b098468e8b56 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:53.984895 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a6b21c-f782-44a8-88de-b098468e8b56 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:54.060751 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a6b21c-f782-44a8-88de-b098468e8b56 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:54.062123 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a6b21c-f782-44a8-88de-b098468e8b56 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:54.074166 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791f335c-1b12-4af4-a3c2-36dda2dd3aab None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:54.123997 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77b23c4-3520-41cb-a58d-84b5717a897e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:54.125327 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77b23c4-3520-41cb-a58d-84b5717a897e None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:54.125958 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77b23c4-3520-41cb-a58d-84b5717a897e None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:54.126384 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77b23c4-3520-41cb-a58d-84b5717a897e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:54.141361 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77b23c4-3520-41cb-a58d-84b5717a897e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 17 19:10:54.152204 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-a77b23c4-3520-41cb-a58d-84b5717a897e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 17 19:10:54.152204 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:54.152204 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:54.152204 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:54.152204 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:54.152204 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:54.152204 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:54.152204 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:54.152204 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:54.152204 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:54.152204 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:54.152204 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:54.152204 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 17 19:10:54.152204 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) May 17 19:10:54.152204 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 17 19:10:54.152204 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 17 19:10:54.153322 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 17 19:10:54.153322 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 17 19:10:54.153322 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 17 19:10:54.153322 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 17 19:10:54.153322 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:54.153322 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:54.153322 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:54.153322 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:54.153322 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:54.153322 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:54.153322 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:54.153322 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:10:54.153322 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:54.153322 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:54.153322 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:54.153322 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:54.154357 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:54.154357 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:54.154357 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:54.154357 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:54.154357 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 17 19:10:54.154357 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 17 19:10:54.154357 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 17 19:10:54.154357 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 17 19:10:54.154357 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 17 19:10:54.154357 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 17 19:10:54.154357 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 17 19:10:54.154357 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:10:54.162106 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 325/637] 10.222.0.58 () {56 vars in 1102 bytes} [Tue May 17 19:10:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:10:54.162106 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77b23c4-3520-41cb-a58d-84b5717a897e None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:54.175113 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 313/638] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 17 19:10:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 17 19:10:54.176165 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a6b21c-f782-44a8-88de-b098468e8b56 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:54.276454 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77b23c4-3520-41cb-a58d-84b5717a897e None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:54.293210 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a6b21c-f782-44a8-88de-b098468e8b56 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:54.308550 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a166cf6a-5132-4889-ae41-2a72bf285215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:54.309341 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a166cf6a-5132-4889-ae41-2a72bf285215 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:54.309635 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a166cf6a-5132-4889-ae41-2a72bf285215 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:54.309896 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a166cf6a-5132-4889-ae41-2a72bf285215 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:54.317987 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a166cf6a-5132-4889-ae41-2a72bf285215 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:54.319387 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a166cf6a-5132-4889-ae41-2a72bf285215 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:54.326580 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983954bf-3eac-487c-bda1-303ffe4b8243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:54.327463 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983954bf-3eac-487c-bda1-303ffe4b8243 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:54.327944 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983954bf-3eac-487c-bda1-303ffe4b8243 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:54.328391 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983954bf-3eac-487c-bda1-303ffe4b8243 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:54.329315 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983954bf-3eac-487c-bda1-303ffe4b8243 None admin] RBAC: Adding query filter params (name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:54.329795 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983954bf-3eac-487c-bda1-303ffe4b8243 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:54.331337 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983954bf-3eac-487c-bda1-303ffe4b8243 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:54.333895 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-a166cf6a-5132-4889-ae41-2a72bf285215 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 17 19:10:54.333895 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:54.333895 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:54.333895 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:54.333895 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:54.333895 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:54.333895 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:54.333895 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:54.333895 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:54.333895 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:54.333895 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:54.333895 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:54.333895 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 17 19:10:54.333895 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_user(user_id) May 17 19:10:54.333895 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 17 19:10:54.333895 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 17 19:10:54.334889 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:54.334889 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:54.334889 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 17 19:10:54.334889 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 17 19:10:54.334889 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 17 19:10:54.334889 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 17 19:10:54.334889 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:54.334889 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:54.334889 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:54.334889 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:54.334889 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:54.334889 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:10:54.334889 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:54.334889 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:54.334889 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:54.334889 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:54.334889 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:54.334889 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:54.335867 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:54.335867 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:54.335867 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 17 19:10:54.335867 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 17 19:10:54.335867 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 17 19:10:54.335867 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 17 19:10:54.335867 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 17 19:10:54.335867 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 17 19:10:54.335867 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 17 19:10:54.335867 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:10:54.335867 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 326/639] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 17 19:10:54 2022] GET /identity/v3/users/glance => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:10:54.335867 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a166cf6a-5132-4889-ae41-2a72bf285215 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:54.346929 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 314/640] 10.222.0.58 () {56 vars in 1103 bytes} [Tue May 17 19:10:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:54.348752 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f8acf9-9cfa-4cdf-b7ea-7a2fb593e23f None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:54.349452 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f8acf9-9cfa-4cdf-b7ea-7a2fb593e23f None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:54.349858 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f8acf9-9cfa-4cdf-b7ea-7a2fb593e23f None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:54.351769 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 315/641] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:10:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:10:54.352741 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f8acf9-9cfa-4cdf-b7ea-7a2fb593e23f None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:54.432037 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a166cf6a-5132-4889-ae41-2a72bf285215 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:54.463995 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f8acf9-9cfa-4cdf-b7ea-7a2fb593e23f None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:54.465361 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11447237-13cf-4ae1-a2bc-3f8447e237b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:54.466149 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11447237-13cf-4ae1-a2bc-3f8447e237b2 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:54.466430 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11447237-13cf-4ae1-a2bc-3f8447e237b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:54.466706 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11447237-13cf-4ae1-a2bc-3f8447e237b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:54.477182 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11447237-13cf-4ae1-a2bc-3f8447e237b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:54.478621 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11447237-13cf-4ae1-a2bc-3f8447e237b2 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:54.495311 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-11447237-13cf-4ae1-a2bc-3f8447e237b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:10:54.495311 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:54.495311 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:54.495311 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:54.495311 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:54.495311 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:54.495311 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:54.495311 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:54.495311 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:54.495311 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:54.495311 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:54.495311 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:54.495311 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 17 19:10:54.495311 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 17 19:10:54.495311 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 17 19:10:54.496324 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 17 19:10:54.496324 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:54.496324 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:54.496324 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:54.496324 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:54.496324 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:54.496324 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:54.496324 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:54.496324 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:10:54.496324 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:54.496324 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:54.496324 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:54.496324 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:54.496324 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:54.496324 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:54.496324 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:54.496324 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:54.497315 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 17 19:10:54.497315 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 17 19:10:54.497315 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:10:54.497315 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:10:54.497315 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 327/642] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:10:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:10:54.497315 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11447237-13cf-4ae1-a2bc-3f8447e237b2 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:54.505262 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fce3897-439a-4048-b963-6722a53ebfca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:54.505934 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fce3897-439a-4048-b963-6722a53ebfca None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:54.506213 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fce3897-439a-4048-b963-6722a53ebfca None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:54.506745 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fce3897-439a-4048-b963-6722a53ebfca None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:54.507317 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fce3897-439a-4048-b963-6722a53ebfca None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:54.507668 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fce3897-439a-4048-b963-6722a53ebfca None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:54.509084 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fce3897-439a-4048-b963-6722a53ebfca None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:54.526821 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 316/643] 10.222.0.58 () {56 vars in 1137 bytes} [Tue May 17 19:10:54 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:54.527919 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7a2992-6025-40f8-a38e-4073140976c2 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:54.528207 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7a2992-6025-40f8-a38e-4073140976c2 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:54.528482 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7a2992-6025-40f8-a38e-4073140976c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:54.544608 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-8e7a2992-6025-40f8-a38e-4073140976c2 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:10:54.546389 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-8e7a2992-6025-40f8-a38e-4073140976c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:10:54.600891 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11447237-13cf-4ae1-a2bc-3f8447e237b2 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:54.615161 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 317/644] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:10:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 17 19:10:54.616380 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7a2992-6025-40f8-a38e-4073140976c2 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:54.639570 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc2c03a-8440-4895-b3c8-de9b23a7a5ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:54.640358 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc2c03a-8440-4895-b3c8-de9b23a7a5ba None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:54.640684 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc2c03a-8440-4895-b3c8-de9b23a7a5ba None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:54.641402 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc2c03a-8440-4895-b3c8-de9b23a7a5ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:54.652313 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc2c03a-8440-4895-b3c8-de9b23a7a5ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:54.653753 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc2c03a-8440-4895-b3c8-de9b23a7a5ba None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:54.670555 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-9bc2c03a-8440-4895-b3c8-de9b23a7a5ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:10:54.670555 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:54.670555 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:54.670555 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:54.670555 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:54.670555 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:54.670555 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:54.670555 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:54.670555 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:54.670555 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:54.670555 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:54.670555 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:54.670555 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 17 19:10:54.670555 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 17 19:10:54.670555 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 17 19:10:54.671554 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 17 19:10:54.671554 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:54.671554 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:54.671554 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:54.671554 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:54.671554 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:54.671554 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:54.671554 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:54.671554 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:10:54.671554 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:54.671554 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:54.671554 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:54.671554 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:54.671554 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:54.671554 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:54.671554 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:54.671554 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:54.672536 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 17 19:10:54.672536 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 17 19:10:54.672536 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:10:54.672536 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:10:54.672536 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 328/645] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:10:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:10:54.672536 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc2c03a-8440-4895-b3c8-de9b23a7a5ba None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:54.728554 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7a2992-6025-40f8-a38e-4073140976c2 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:54.762117 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c804e3f0-5d13-4d31-8cab-c0e8d0687abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:54.762773 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c804e3f0-5d13-4d31-8cab-c0e8d0687abe None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:54.763052 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c804e3f0-5d13-4d31-8cab-c0e8d0687abe None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:54.763322 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c804e3f0-5d13-4d31-8cab-c0e8d0687abe None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:54.764066 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c804e3f0-5d13-4d31-8cab-c0e8d0687abe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:54.764363 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c804e3f0-5d13-4d31-8cab-c0e8d0687abe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:54.765734 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c804e3f0-5d13-4d31-8cab-c0e8d0687abe None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:54.776663 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 318/646] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 17 19:10:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:54.777457 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c804e3f0-5d13-4d31-8cab-c0e8d0687abe None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:54.790339 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc2c03a-8440-4895-b3c8-de9b23a7a5ba None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:54.825380 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de02bd22-5afa-4449-b0f3-a2ab86b860ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:54.826026 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de02bd22-5afa-4449-b0f3-a2ab86b860ea None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:54.826315 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de02bd22-5afa-4449-b0f3-a2ab86b860ea None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:54.826573 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de02bd22-5afa-4449-b0f3-a2ab86b860ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:54.879258 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c804e3f0-5d13-4d31-8cab-c0e8d0687abe None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:54.895798 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de02bd22-5afa-4449-b0f3-a2ab86b860ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:54.897359 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de02bd22-5afa-4449-b0f3-a2ab86b860ea None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:54.912680 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf0bf46-0fbd-44b1-92a8-43b498944eb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:54.913492 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf0bf46-0fbd-44b1-92a8-43b498944eb3 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:54.913769 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf0bf46-0fbd-44b1-92a8-43b498944eb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:54.914040 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf0bf46-0fbd-44b1-92a8-43b498944eb3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:54.925568 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf0bf46-0fbd-44b1-92a8-43b498944eb3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 17 19:10:54.934110 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-9cf0bf46-0fbd-44b1-92a8-43b498944eb3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 17 19:10:54.934110 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:54.934110 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:54.934110 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:54.934110 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:54.934110 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:54.934110 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:54.934110 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:54.934110 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:54.934110 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:54.934110 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:54.934110 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:54.934110 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 17 19:10:54.934110 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) May 17 19:10:54.934110 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 17 19:10:54.934110 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 17 19:10:54.935129 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 17 19:10:54.935129 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 17 19:10:54.935129 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 17 19:10:54.935129 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 17 19:10:54.935129 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:54.935129 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:54.935129 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:54.935129 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:54.935129 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:54.935129 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:54.935129 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:54.935129 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:10:54.935129 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:54.935129 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:54.935129 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:54.935129 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:54.941209 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:54.941209 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:54.941209 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:54.941209 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:54.941209 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 17 19:10:54.941209 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 17 19:10:54.941209 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 17 19:10:54.941209 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 17 19:10:54.941209 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 17 19:10:54.941209 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 17 19:10:54.941209 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 17 19:10:54.941209 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:10:54.941209 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 319/647] 10.222.0.58 () {56 vars in 1102 bytes} [Tue May 17 19:10:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:10:54.941209 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf0bf46-0fbd-44b1-92a8-43b498944eb3 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:55.012526 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 329/648] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 17 19:10:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 17 19:10:55.013524 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de02bd22-5afa-4449-b0f3-a2ab86b860ea None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:55.042806 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf0bf46-0fbd-44b1-92a8-43b498944eb3 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:55.075061 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d99cefb-be51-43cf-af34-9e470ab72e0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:55.075747 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d99cefb-be51-43cf-af34-9e470ab72e0b None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:55.076019 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d99cefb-be51-43cf-af34-9e470ab72e0b None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:55.076285 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d99cefb-be51-43cf-af34-9e470ab72e0b None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:55.076988 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d99cefb-be51-43cf-af34-9e470ab72e0b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:55.077280 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d99cefb-be51-43cf-af34-9e470ab72e0b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:55.078636 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d99cefb-be51-43cf-af34-9e470ab72e0b None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:55.094203 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 320/649] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 17 19:10:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:55.094203 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d99cefb-be51-43cf-af34-9e470ab72e0b None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:55.112783 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de02bd22-5afa-4449-b0f3-a2ab86b860ea None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:55.144701 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298afcda-01e0-403b-8da6-ff48a47db66c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:55.145461 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298afcda-01e0-403b-8da6-ff48a47db66c None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:55.145735 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298afcda-01e0-403b-8da6-ff48a47db66c None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:55.145997 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298afcda-01e0-403b-8da6-ff48a47db66c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:55.152301 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298afcda-01e0-403b-8da6-ff48a47db66c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:55.153729 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298afcda-01e0-403b-8da6-ff48a47db66c None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:55.167317 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-298afcda-01e0-403b-8da6-ff48a47db66c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 17 19:10:55.167317 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:55.167317 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:55.167317 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:55.167317 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:55.167317 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:55.167317 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:55.167317 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:55.167317 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:55.167317 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:55.167317 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:55.167317 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:55.167317 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 17 19:10:55.167317 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(project_id) May 17 19:10:55.167317 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 17 19:10:55.168407 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 17 19:10:55.168407 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:55.168407 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:55.168407 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:55.168407 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:55.168407 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:55.168407 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:55.168407 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:55.168407 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:10:55.168407 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:55.168407 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:55.168407 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:55.168407 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:55.168407 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:55.168407 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:55.168407 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:55.168407 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:55.169663 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 17 19:10:55.169663 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 17 19:10:55.169663 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 17 19:10:55.169663 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 17 19:10:55.169663 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 17 19:10:55.169663 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 17 19:10:55.169663 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 17 19:10:55.169663 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:10:55.169663 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 330/650] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 17 19:10:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:10:55.169663 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298afcda-01e0-403b-8da6-ff48a47db66c None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:55.187199 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d99cefb-be51-43cf-af34-9e470ab72e0b None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:55.218401 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d787cf95-32e4-4236-8b57-1fe9da696893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:55.219048 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d787cf95-32e4-4236-8b57-1fe9da696893 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:55.219323 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d787cf95-32e4-4236-8b57-1fe9da696893 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:55.219630 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d787cf95-32e4-4236-8b57-1fe9da696893 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:55.220350 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d787cf95-32e4-4236-8b57-1fe9da696893 None admin] RBAC: Adding query filter params (name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:55.220648 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d787cf95-32e4-4236-8b57-1fe9da696893 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:55.222285 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d787cf95-32e4-4236-8b57-1fe9da696893 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:55.230195 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 321/651] 10.222.0.58 () {56 vars in 1103 bytes} [Tue May 17 19:10:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:55.230984 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d787cf95-32e4-4236-8b57-1fe9da696893 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:55.261115 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298afcda-01e0-403b-8da6-ff48a47db66c None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:55.292022 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b643ffd5-bd48-48d8-a888-2f939885c1db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:55.292790 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b643ffd5-bd48-48d8-a888-2f939885c1db None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:55.293063 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b643ffd5-bd48-48d8-a888-2f939885c1db None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:55.293328 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b643ffd5-bd48-48d8-a888-2f939885c1db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:55.303904 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b643ffd5-bd48-48d8-a888-2f939885c1db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:55.305322 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b643ffd5-bd48-48d8-a888-2f939885c1db None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:55.321358 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-b643ffd5-bd48-48d8-a888-2f939885c1db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:10:55.321358 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:55.321358 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:55.321358 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:55.321358 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:55.321358 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:55.321358 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:55.321358 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:55.321358 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:55.321358 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:55.321358 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:55.321358 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:55.321358 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 17 19:10:55.321358 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 17 19:10:55.321358 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 17 19:10:55.322274 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 17 19:10:55.322274 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:55.322274 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:55.322274 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:55.322274 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:55.322274 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:55.322274 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:55.322274 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:55.322274 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:10:55.322274 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:55.322274 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:55.322274 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:55.322274 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:55.322274 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:55.322274 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:55.322274 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:55.322274 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:55.323214 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 17 19:10:55.323214 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 17 19:10:55.323214 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:10:55.323214 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:10:55.323214 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 331/652] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:10:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:10:55.323893 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d787cf95-32e4-4236-8b57-1fe9da696893 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:55.324019 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aed7b45-bb00-48c4-808f-cc3541920c09 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:55.324284 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aed7b45-bb00-48c4-808f-cc3541920c09 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:55.324554 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aed7b45-bb00-48c4-808f-cc3541920c09 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:55.327532 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 332/653] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:10:55 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:10:55.327532 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aed7b45-bb00-48c4-808f-cc3541920c09 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:55.359319 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b40ed9f-e181-4643-bc35-5c7840ad2b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:55.360129 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b40ed9f-e181-4643-bc35-5c7840ad2b55 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:55.360406 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b40ed9f-e181-4643-bc35-5c7840ad2b55 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:55.360676 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b40ed9f-e181-4643-bc35-5c7840ad2b55 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:55.367931 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b40ed9f-e181-4643-bc35-5c7840ad2b55 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:55.369351 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b40ed9f-e181-4643-bc35-5c7840ad2b55 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:55.382938 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-4b40ed9f-e181-4643-bc35-5c7840ad2b55 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 17 19:10:55.382938 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:55.382938 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:55.382938 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:55.382938 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:55.382938 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:55.382938 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:55.382938 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:55.382938 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:55.382938 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:55.382938 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:55.382938 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:55.382938 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 17 19:10:55.382938 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_user(user_id) May 17 19:10:55.382938 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 17 19:10:55.382938 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 17 19:10:55.383901 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:55.383901 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:55.383901 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 17 19:10:55.383901 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 17 19:10:55.383901 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 17 19:10:55.383901 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 17 19:10:55.383901 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:55.383901 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:55.383901 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:55.383901 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:55.383901 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:55.383901 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:10:55.383901 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:55.383901 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:55.383901 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:55.383901 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:55.383901 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:55.383901 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:55.384842 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:55.384842 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:55.384842 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 17 19:10:55.384842 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 17 19:10:55.384842 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 17 19:10:55.384842 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 17 19:10:55.384842 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 17 19:10:55.384842 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 17 19:10:55.384842 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 17 19:10:55.384842 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:10:55.384842 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 322/654] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 17 19:10:55 2022] GET /identity/v3/users/cinder => generated 84 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:10:55.385438 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b40ed9f-e181-4643-bc35-5c7840ad2b55 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:55.426872 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aed7b45-bb00-48c4-808f-cc3541920c09 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:55.458167 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c5876e-3f09-42e7-849c-0419237f1fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:55.458805 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c5876e-3f09-42e7-849c-0419237f1fbf None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:55.459079 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c5876e-3f09-42e7-849c-0419237f1fbf None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:55.459346 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c5876e-3f09-42e7-849c-0419237f1fbf None admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:55.460152 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c5876e-3f09-42e7-849c-0419237f1fbf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:55.460442 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c5876e-3f09-42e7-849c-0419237f1fbf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:55.461860 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c5876e-3f09-42e7-849c-0419237f1fbf None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:55.473770 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 333/655] 10.222.0.58 () {56 vars in 1148 bytes} [Tue May 17 19:10:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:55.474580 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c5876e-3f09-42e7-849c-0419237f1fbf None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:55.478439 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b40ed9f-e181-4643-bc35-5c7840ad2b55 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:55.509571 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b598965-252f-43b1-9fbd-ea28015d105f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:55.510346 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b598965-252f-43b1-9fbd-ea28015d105f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:55.510622 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b598965-252f-43b1-9fbd-ea28015d105f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:55.510890 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b598965-252f-43b1-9fbd-ea28015d105f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:55.521415 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b598965-252f-43b1-9fbd-ea28015d105f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:55.522839 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b598965-252f-43b1-9fbd-ea28015d105f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:55.539677 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-5b598965-252f-43b1-9fbd-ea28015d105f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:10:55.539677 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:55.539677 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:55.539677 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:55.539677 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:55.539677 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:55.539677 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:55.539677 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:55.539677 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:55.539677 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:55.539677 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:55.539677 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:55.539677 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 17 19:10:55.539677 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 17 19:10:55.539677 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 17 19:10:55.540531 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 17 19:10:55.540531 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:55.540531 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:55.540531 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:55.540531 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:55.540531 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:55.540531 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:55.540531 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:55.540531 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:10:55.540531 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:55.540531 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:55.540531 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:55.540531 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:55.540531 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:55.540531 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:55.540531 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:55.540531 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:55.541438 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 17 19:10:55.541438 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 17 19:10:55.541438 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:10:55.541438 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:10:55.541438 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 323/656] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:10:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:10:55.542251 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5cfd70-6169-471e-a516-1dd26693fc85 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:55.542524 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5cfd70-6169-471e-a516-1dd26693fc85 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:55.542791 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5cfd70-6169-471e-a516-1dd26693fc85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:55.559613 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-8c5cfd70-6169-471e-a516-1dd26693fc85 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:10:55.560762 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-8c5cfd70-6169-471e-a516-1dd26693fc85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:10:55.568856 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c5876e-3f09-42e7-849c-0419237f1fbf None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:55.601569 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edc6081-814c-4fc2-bc94-be1a974c9da8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:55.602223 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edc6081-814c-4fc2-bc94-be1a974c9da8 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:55.602502 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edc6081-814c-4fc2-bc94-be1a974c9da8 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:55.602778 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edc6081-814c-4fc2-bc94-be1a974c9da8 None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:55.603504 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edc6081-814c-4fc2-bc94-be1a974c9da8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:55.603818 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edc6081-814c-4fc2-bc94-be1a974c9da8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:55.605177 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edc6081-814c-4fc2-bc94-be1a974c9da8 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:55.615790 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 334/657] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 17 19:10:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:55.616560 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edc6081-814c-4fc2-bc94-be1a974c9da8 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:55.624040 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 324/658] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:10:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 17 19:10:55.624862 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5cfd70-6169-471e-a516-1dd26693fc85 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:55.712778 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edc6081-814c-4fc2-bc94-be1a974c9da8 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:55.722335 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5cfd70-6169-471e-a516-1dd26693fc85 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:55.743967 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9893988-19c8-46d2-ab7a-476b1ec1e6ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:55.744617 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9893988-19c8-46d2-ab7a-476b1ec1e6ec None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:55.744894 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9893988-19c8-46d2-ab7a-476b1ec1e6ec None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:55.745166 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9893988-19c8-46d2-ab7a-476b1ec1e6ec None admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:55.745986 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9893988-19c8-46d2-ab7a-476b1ec1e6ec None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:55.746288 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9893988-19c8-46d2-ab7a-476b1ec1e6ec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:55.747729 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9893988-19c8-46d2-ab7a-476b1ec1e6ec None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:55.753525 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593f1340-17ea-4deb-832d-ba43696c6e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:55.754681 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593f1340-17ea-4deb-832d-ba43696c6e71 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:55.754959 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593f1340-17ea-4deb-832d-ba43696c6e71 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:55.755227 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593f1340-17ea-4deb-832d-ba43696c6e71 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:55.756141 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593f1340-17ea-4deb-832d-ba43696c6e71 None admin] RBAC: Adding query filter params (role.id=1754a95b778c4e20b53b20cb52ada92b, scope.project.id=1a9d3a0631284c9fb113266036993e2d, user.id=65c1f878b0dc4d63a4f5dcb12fac1dc1) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:55.756443 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593f1340-17ea-4deb-832d-ba43696c6e71 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:55.757849 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593f1340-17ea-4deb-832d-ba43696c6e71 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:55.763530 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 325/659] 10.222.0.58 () {56 vars in 1374 bytes} [Tue May 17 19:10:55 2022] GET /identity/v3/role_assignments?user.id=65c1f878b0dc4d63a4f5dcb12fac1dc1&scope.project.id=1a9d3a0631284c9fb113266036993e2d&role.id=1754a95b778c4e20b53b20cb52ada92b => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:55.764215 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593f1340-17ea-4deb-832d-ba43696c6e71 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:55.764306 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 335/660] 10.222.0.58 () {56 vars in 1137 bytes} [Tue May 17 19:10:55 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:55.765078 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9893988-19c8-46d2-ab7a-476b1ec1e6ec None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:55.873790 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593f1340-17ea-4deb-832d-ba43696c6e71 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:55.880511 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9893988-19c8-46d2-ab7a-476b1ec1e6ec None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:55.909286 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6f19ec-fdf9-4bf4-a5d0-d625bb8af920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:55.909953 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6f19ec-fdf9-4bf4-a5d0-d625bb8af920 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:55.910261 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6f19ec-fdf9-4bf4-a5d0-d625bb8af920 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:55.910532 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6f19ec-fdf9-4bf4-a5d0-d625bb8af920 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:55.911249 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6f19ec-fdf9-4bf4-a5d0-d625bb8af920 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:55.911581 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6f19ec-fdf9-4bf4-a5d0-d625bb8af920 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:55.912961 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6f19ec-fdf9-4bf4-a5d0-d625bb8af920 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:55.916323 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e176797-059f-45e0-84cd-82def174f3b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:55.917113 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e176797-059f-45e0-84cd-82def174f3b0 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:55.917389 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e176797-059f-45e0-84cd-82def174f3b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:55.917656 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e176797-059f-45e0-84cd-82def174f3b0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:55.928856 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e176797-059f-45e0-84cd-82def174f3b0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:55.929049 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 326/661] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 17 19:10:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:55.929049 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6f19ec-fdf9-4bf4-a5d0-d625bb8af920 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:55.929215 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e176797-059f-45e0-84cd-82def174f3b0 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:55.949702 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-6e176797-059f-45e0-84cd-82def174f3b0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 17 19:10:55.949702 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:55.949702 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:55.949702 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:55.949702 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:55.949702 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:55.949702 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:55.949702 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:55.949702 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:55.949702 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:55.949702 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:55.949702 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:55.949702 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 17 19:10:55.949702 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(project_id) May 17 19:10:55.949702 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 17 19:10:55.950824 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 17 19:10:55.950824 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:55.950824 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:55.950824 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:55.950824 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:55.950824 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:55.950824 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:55.950824 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:55.950824 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:10:55.950824 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:55.950824 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:55.950824 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:55.950824 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:55.950824 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:55.950824 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:55.950824 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:55.950824 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:55.951844 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 17 19:10:55.951844 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 17 19:10:55.951844 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 17 19:10:55.951844 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 17 19:10:55.951844 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 17 19:10:55.951844 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 17 19:10:55.951844 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 17 19:10:55.951844 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:10:55.951844 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 336/662] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 17 19:10:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:10:55.951844 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e176797-059f-45e0-84cd-82def174f3b0 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:56.040405 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6f19ec-fdf9-4bf4-a5d0-d625bb8af920 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:56.067465 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e176797-059f-45e0-84cd-82def174f3b0 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:56.074095 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c062b3-41b1-4fe0-a4a1-e02b0d846145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:56.074887 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c062b3-41b1-4fe0-a4a1-e02b0d846145 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:56.075161 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c062b3-41b1-4fe0-a4a1-e02b0d846145 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:56.075427 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c062b3-41b1-4fe0-a4a1-e02b0d846145 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:56.085394 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c062b3-41b1-4fe0-a4a1-e02b0d846145 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 17 19:10:56.092911 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-a2c062b3-41b1-4fe0-a4a1-e02b0d846145 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 17 19:10:56.092911 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:56.092911 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:56.092911 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:56.092911 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:56.092911 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:56.092911 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:56.092911 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:56.092911 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:56.092911 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:56.092911 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:56.092911 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:56.092911 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 17 19:10:56.092911 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) May 17 19:10:56.092911 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 17 19:10:56.092911 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 17 19:10:56.094058 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 17 19:10:56.094058 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 17 19:10:56.094058 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 17 19:10:56.094058 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 17 19:10:56.094058 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:56.094058 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:56.094058 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:56.094058 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:56.094058 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:56.094058 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:56.094058 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:56.094058 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:10:56.094058 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:56.094058 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:56.094058 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:56.094058 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:56.094919 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:56.094919 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:56.094919 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:56.094919 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:56.094919 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 17 19:10:56.094919 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 17 19:10:56.094919 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 17 19:10:56.094919 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 17 19:10:56.094919 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 17 19:10:56.094919 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 17 19:10:56.094919 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 17 19:10:56.094919 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:10:56.094919 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 327/663] 10.222.0.58 () {56 vars in 1102 bytes} [Tue May 17 19:10:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:10:56.094919 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c062b3-41b1-4fe0-a4a1-e02b0d846145 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:56.106223 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5686bc92-3b45-4a7b-902a-9671e2829b0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:56.107217 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5686bc92-3b45-4a7b-902a-9671e2829b0b None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:56.108044 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5686bc92-3b45-4a7b-902a-9671e2829b0b None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:56.108508 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5686bc92-3b45-4a7b-902a-9671e2829b0b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:56.180021 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5686bc92-3b45-4a7b-902a-9671e2829b0b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:56.181468 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5686bc92-3b45-4a7b-902a-9671e2829b0b None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:56.194732 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c062b3-41b1-4fe0-a4a1-e02b0d846145 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:56.227921 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73527c6b-b483-427f-b1dc-e11d25784ab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:56.228717 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73527c6b-b483-427f-b1dc-e11d25784ab3 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:56.229008 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73527c6b-b483-427f-b1dc-e11d25784ab3 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:56.229264 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73527c6b-b483-427f-b1dc-e11d25784ab3 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:56.236763 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73527c6b-b483-427f-b1dc-e11d25784ab3 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:56.238219 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73527c6b-b483-427f-b1dc-e11d25784ab3 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:56.253050 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-73527c6b-b483-427f-b1dc-e11d25784ab3 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 17 19:10:56.253050 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:56.253050 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:56.253050 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:56.253050 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:56.253050 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:56.253050 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:56.253050 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:56.253050 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:56.253050 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:56.253050 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:56.253050 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:56.253050 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 17 19:10:56.253050 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_user(user_id) May 17 19:10:56.253050 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 17 19:10:56.253050 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 17 19:10:56.254739 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:56.254739 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:56.254739 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 17 19:10:56.254739 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 17 19:10:56.254739 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 17 19:10:56.254739 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 17 19:10:56.254739 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:56.254739 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:56.254739 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:56.254739 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:56.254739 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:56.254739 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:10:56.254739 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:56.254739 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:56.254739 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:56.254739 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:56.254739 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:56.254739 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:56.255732 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:56.255732 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:56.255732 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 17 19:10:56.255732 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 17 19:10:56.255732 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 17 19:10:56.255732 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 17 19:10:56.255732 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 17 19:10:56.255732 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 17 19:10:56.255732 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 17 19:10:56.255732 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:10:56.255732 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 328/664] 10.222.0.58 () {56 vars in 1102 bytes} [Tue May 17 19:10:56 2022] GET /identity/v3/users/neutron => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:10:56.255732 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73527c6b-b483-427f-b1dc-e11d25784ab3 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:56.290521 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 337/665] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 17 19:10:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 17 19:10:56.291299 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5686bc92-3b45-4a7b-902a-9671e2829b0b None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:56.357901 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73527c6b-b483-427f-b1dc-e11d25784ab3 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:56.388263 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5686bc92-3b45-4a7b-902a-9671e2829b0b None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:56.391277 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d1e4ce-0857-4aa1-b7c8-a73a27b67a4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:56.391960 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d1e4ce-0857-4aa1-b7c8-a73a27b67a4f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:56.392238 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d1e4ce-0857-4aa1-b7c8-a73a27b67a4f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:56.392510 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d1e4ce-0857-4aa1-b7c8-a73a27b67a4f None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:56.393289 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d1e4ce-0857-4aa1-b7c8-a73a27b67a4f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:56.393594 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d1e4ce-0857-4aa1-b7c8-a73a27b67a4f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:56.395061 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d1e4ce-0857-4aa1-b7c8-a73a27b67a4f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:56.412381 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 329/666] 10.222.0.58 () {56 vars in 1148 bytes} [Tue May 17 19:10:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:56.412381 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d1e4ce-0857-4aa1-b7c8-a73a27b67a4f None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:56.422515 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815421cb-4aae-4977-b680-1ab437a2921a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:56.423316 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815421cb-4aae-4977-b680-1ab437a2921a None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:56.423638 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815421cb-4aae-4977-b680-1ab437a2921a None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:56.423915 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815421cb-4aae-4977-b680-1ab437a2921a None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:56.424692 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815421cb-4aae-4977-b680-1ab437a2921a None admin] RBAC: Adding query filter params (name=service) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:56.425017 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815421cb-4aae-4977-b680-1ab437a2921a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:56.427014 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815421cb-4aae-4977-b680-1ab437a2921a None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:56.439174 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 338/667] 10.222.0.58 () {56 vars in 1103 bytes} [Tue May 17 19:10:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:56.439174 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815421cb-4aae-4977-b680-1ab437a2921a None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:56.515789 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d1e4ce-0857-4aa1-b7c8-a73a27b67a4f None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:56.534538 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815421cb-4aae-4977-b680-1ab437a2921a None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:56.548530 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b35fac4-362d-40da-a0ac-09fe5d21a063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:56.549132 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b35fac4-362d-40da-a0ac-09fe5d21a063 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:56.549493 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b35fac4-362d-40da-a0ac-09fe5d21a063 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:56.549769 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b35fac4-362d-40da-a0ac-09fe5d21a063 None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:56.550589 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b35fac4-362d-40da-a0ac-09fe5d21a063 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:56.551528 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b35fac4-362d-40da-a0ac-09fe5d21a063 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:56.553014 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b35fac4-362d-40da-a0ac-09fe5d21a063 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:56.569933 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 330/668] 10.222.0.58 () {56 vars in 1139 bytes} [Tue May 17 19:10:56 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:56.571049 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b35fac4-362d-40da-a0ac-09fe5d21a063 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:56.575233 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b10d68e-601c-4df9-98a2-f806d0f4eeb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:56.576092 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b10d68e-601c-4df9-98a2-f806d0f4eeb3 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:56.576375 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b10d68e-601c-4df9-98a2-f806d0f4eeb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:56.576786 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b10d68e-601c-4df9-98a2-f806d0f4eeb3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:56.587352 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b10d68e-601c-4df9-98a2-f806d0f4eeb3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:56.588842 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b10d68e-601c-4df9-98a2-f806d0f4eeb3 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:56.606271 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-2b10d68e-601c-4df9-98a2-f806d0f4eeb3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:10:56.606271 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:56.606271 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:56.606271 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:56.606271 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:56.606271 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:56.606271 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:56.606271 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:56.606271 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:56.606271 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:56.606271 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:56.606271 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:56.606271 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 17 19:10:56.606271 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 17 19:10:56.606271 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 17 19:10:56.607245 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 17 19:10:56.607245 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:56.607245 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:56.607245 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:56.607245 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:56.607245 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:56.607245 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:56.607245 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:56.607245 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:10:56.607245 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:56.607245 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:56.607245 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:56.607245 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:56.607245 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:56.607245 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:56.607245 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:56.607245 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:56.608179 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 17 19:10:56.608179 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 17 19:10:56.608179 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:10:56.608179 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:10:56.608179 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 339/669] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:10:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:10:56.608451 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b10d68e-601c-4df9-98a2-f806d0f4eeb3 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:56.674635 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b35fac4-362d-40da-a0ac-09fe5d21a063 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:56.706751 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b10d68e-601c-4df9-98a2-f806d0f4eeb3 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:56.707643 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749b01e0-b97f-4f25-bd16-d6e9e056714e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:56.708503 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749b01e0-b97f-4f25-bd16-d6e9e056714e None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:56.708989 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749b01e0-b97f-4f25-bd16-d6e9e056714e None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:56.709321 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749b01e0-b97f-4f25-bd16-d6e9e056714e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:56.710201 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749b01e0-b97f-4f25-bd16-d6e9e056714e None admin] RBAC: Adding query filter params (role.id=1754a95b778c4e20b53b20cb52ada92b, scope.project.id=1a9d3a0631284c9fb113266036993e2d, user.id=57e888884d2b4629bfd12d1034ce5be4) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:56.710595 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749b01e0-b97f-4f25-bd16-d6e9e056714e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:56.712120 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749b01e0-b97f-4f25-bd16-d6e9e056714e None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:56.727748 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 331/670] 10.222.0.58 () {56 vars in 1374 bytes} [Tue May 17 19:10:56 2022] GET /identity/v3/role_assignments?user.id=57e888884d2b4629bfd12d1034ce5be4&scope.project.id=1a9d3a0631284c9fb113266036993e2d&role.id=1754a95b778c4e20b53b20cb52ada92b => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:56.727748 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749b01e0-b97f-4f25-bd16-d6e9e056714e None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:56.752015 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd4cd85-e4e6-45ea-bf96-f655b575d63a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:56.752995 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd4cd85-e4e6-45ea-bf96-f655b575d63a None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:56.753378 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd4cd85-e4e6-45ea-bf96-f655b575d63a None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:56.753745 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd4cd85-e4e6-45ea-bf96-f655b575d63a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:56.765670 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd4cd85-e4e6-45ea-bf96-f655b575d63a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:56.767373 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd4cd85-e4e6-45ea-bf96-f655b575d63a None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:56.786245 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-fdd4cd85-e4e6-45ea-bf96-f655b575d63a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:10:56.786245 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:56.786245 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:56.786245 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:56.786245 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:56.786245 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:56.786245 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:56.786245 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:56.786245 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:56.786245 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:56.786245 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:56.786245 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:56.786245 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 17 19:10:56.786245 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 17 19:10:56.786245 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 17 19:10:56.787427 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 17 19:10:56.787427 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:56.787427 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:56.787427 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:56.787427 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:56.787427 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:56.787427 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:56.787427 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:56.787427 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:10:56.787427 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:56.787427 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:56.787427 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:56.787427 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:56.787427 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:56.787427 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:56.787427 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:56.787427 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:56.788377 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 17 19:10:56.788377 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 17 19:10:56.788377 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:10:56.788377 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:10:56.791514 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 340/671] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:10:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:10:56.791514 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd4cd85-e4e6-45ea-bf96-f655b575d63a None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:56.829138 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749b01e0-b97f-4f25-bd16-d6e9e056714e None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:56.865251 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c95fed1-2df6-4179-97a8-101120e0d3c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:56.865948 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c95fed1-2df6-4179-97a8-101120e0d3c9 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:56.866228 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c95fed1-2df6-4179-97a8-101120e0d3c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:56.866564 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c95fed1-2df6-4179-97a8-101120e0d3c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:56.909750 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd4cd85-e4e6-45ea-bf96-f655b575d63a None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:56.943375 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c95fed1-2df6-4179-97a8-101120e0d3c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:56.944920 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c95fed1-2df6-4179-97a8-101120e0d3c9 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:56.953249 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec482f06-300b-4203-98f3-78750c26bac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:56.953941 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec482f06-300b-4203-98f3-78750c26bac8 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:56.954224 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec482f06-300b-4203-98f3-78750c26bac8 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:56.955144 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec482f06-300b-4203-98f3-78750c26bac8 None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:56.956476 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec482f06-300b-4203-98f3-78750c26bac8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:56.956476 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec482f06-300b-4203-98f3-78750c26bac8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:56.958695 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec482f06-300b-4203-98f3-78750c26bac8 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:56.974212 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 341/672] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 17 19:10:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:56.975413 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec482f06-300b-4203-98f3-78750c26bac8 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:57.077848 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 332/673] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 17 19:10:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 17 19:10:57.083548 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c95fed1-2df6-4179-97a8-101120e0d3c9 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:57.132406 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec482f06-300b-4203-98f3-78750c26bac8 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:57.209567 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d9b1b7-cf60-4532-9c8a-98a266b17db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:57.210229 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d9b1b7-cf60-4532-9c8a-98a266b17db0 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:57.210510 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d9b1b7-cf60-4532-9c8a-98a266b17db0 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:57.210780 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d9b1b7-cf60-4532-9c8a-98a266b17db0 None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:57.211514 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d9b1b7-cf60-4532-9c8a-98a266b17db0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:57.211822 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d9b1b7-cf60-4532-9c8a-98a266b17db0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:57.213236 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d9b1b7-cf60-4532-9c8a-98a266b17db0 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:57.241852 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 342/674] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 17 19:10:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:57.242711 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d9b1b7-cf60-4532-9c8a-98a266b17db0 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:57.260022 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c95fed1-2df6-4179-97a8-101120e0d3c9 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:57.305783 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80464934-7d00-4a3b-ad19-53d3f93a71cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:57.308706 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80464934-7d00-4a3b-ad19-53d3f93a71cb None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:57.308706 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80464934-7d00-4a3b-ad19-53d3f93a71cb None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:57.308706 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80464934-7d00-4a3b-ad19-53d3f93a71cb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:57.316977 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80464934-7d00-4a3b-ad19-53d3f93a71cb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:57.318484 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80464934-7d00-4a3b-ad19-53d3f93a71cb None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:57.335133 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-80464934-7d00-4a3b-ad19-53d3f93a71cb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 17 19:10:57.335133 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:57.335133 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:57.335133 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:57.335133 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:57.335133 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:57.335133 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:57.335133 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:57.335133 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:57.335133 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:57.335133 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:57.335133 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:57.335133 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 17 19:10:57.335133 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(project_id) May 17 19:10:57.335133 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 17 19:10:57.336095 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 17 19:10:57.336095 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:57.336095 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:57.336095 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:57.336095 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:57.336095 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:57.336095 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:57.336095 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:57.336095 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:10:57.336095 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:57.336095 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:57.336095 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:57.336095 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:57.336095 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:57.336095 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:57.336095 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:57.336095 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:57.337051 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 17 19:10:57.337051 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 17 19:10:57.337051 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 17 19:10:57.337051 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 17 19:10:57.337051 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 17 19:10:57.337051 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 17 19:10:57.337051 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 17 19:10:57.337051 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:10:57.343514 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 333/675] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 17 19:10:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:10:57.343514 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80464934-7d00-4a3b-ad19-53d3f93a71cb None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:57.462989 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80464934-7d00-4a3b-ad19-53d3f93a71cb None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:57.466054 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d9b1b7-cf60-4532-9c8a-98a266b17db0 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:57.507159 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2d1b16-a1c0-4aa1-a7a4-cf6d98b99d01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:57.508499 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2d1b16-a1c0-4aa1-a7a4-cf6d98b99d01 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:57.508779 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2d1b16-a1c0-4aa1-a7a4-cf6d98b99d01 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:57.509047 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2d1b16-a1c0-4aa1-a7a4-cf6d98b99d01 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:57.516991 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2d1b16-a1c0-4aa1-a7a4-cf6d98b99d01 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:57.518902 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2d1b16-a1c0-4aa1-a7a4-cf6d98b99d01 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:57.535688 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c2ab35-a4b1-4049-83cb-cf2adb9c62ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:57.536864 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c2ab35-a4b1-4049-83cb-cf2adb9c62ed None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:57.537387 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c2ab35-a4b1-4049-83cb-cf2adb9c62ed None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:57.537592 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-fc2d1b16-a1c0-4aa1-a7a4-cf6d98b99d01 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 17 19:10:57.537592 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:57.537592 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:57.537592 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:57.537592 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:57.537592 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:57.537592 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:57.537592 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:57.537592 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:57.537592 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:57.537592 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:57.537592 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:57.537592 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 17 19:10:57.537592 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_user(user_id) May 17 19:10:57.537592 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 17 19:10:57.537592 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 17 19:10:57.538503 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:57.538503 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:57.538503 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 17 19:10:57.538503 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 17 19:10:57.538503 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 17 19:10:57.538503 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 17 19:10:57.538503 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:57.538503 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:57.538503 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:57.538503 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:57.538503 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:57.538503 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:10:57.538503 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:57.538503 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:57.538503 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:57.538503 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:57.538503 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:57.538503 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:57.539461 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:57.539461 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:57.539461 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 17 19:10:57.539461 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 17 19:10:57.539461 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 17 19:10:57.539461 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 17 19:10:57.539461 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 17 19:10:57.539461 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 17 19:10:57.539461 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 17 19:10:57.539461 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:10:57.540800 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c2ab35-a4b1-4049-83cb-cf2adb9c62ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:57.543405 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 334/676] 10.222.0.58 () {56 vars in 1096 bytes} [Tue May 17 19:10:57 2022] GET /identity/v3/users/swift => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:10:57.544372 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2d1b16-a1c0-4aa1-a7a4-cf6d98b99d01 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:57.555014 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c2ab35-a4b1-4049-83cb-cf2adb9c62ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:57.556805 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c2ab35-a4b1-4049-83cb-cf2adb9c62ed None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:57.574910 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-83c2ab35-a4b1-4049-83cb-cf2adb9c62ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:10:57.574910 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:57.574910 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:57.574910 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:57.574910 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:57.574910 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:57.574910 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:57.574910 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:57.574910 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:57.574910 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:57.574910 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:57.574910 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:57.574910 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 17 19:10:57.574910 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 17 19:10:57.574910 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 17 19:10:57.575948 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 17 19:10:57.575948 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:57.575948 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:57.575948 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:57.575948 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:57.575948 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:57.575948 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:57.575948 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:57.575948 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:10:57.575948 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:57.575948 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:57.575948 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:57.575948 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:57.575948 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:57.575948 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:57.575948 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:57.575948 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:57.576827 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 17 19:10:57.576827 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 17 19:10:57.576827 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:10:57.576827 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:10:57.583228 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 343/677] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:10:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:10:57.584315 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c2ab35-a4b1-4049-83cb-cf2adb9c62ed None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:57.668716 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2d1b16-a1c0-4aa1-a7a4-cf6d98b99d01 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:57.688353 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c2ab35-a4b1-4049-83cb-cf2adb9c62ed None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:57.708684 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14507de-abe5-4707-bc72-e07633b6ae7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:57.709679 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14507de-abe5-4707-bc72-e07633b6ae7a None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:57.710157 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14507de-abe5-4707-bc72-e07633b6ae7a None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:57.710632 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14507de-abe5-4707-bc72-e07633b6ae7a None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:57.711653 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14507de-abe5-4707-bc72-e07633b6ae7a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:57.712159 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14507de-abe5-4707-bc72-e07633b6ae7a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:57.713784 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14507de-abe5-4707-bc72-e07633b6ae7a None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:57.726654 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 335/678] 10.222.0.58 () {56 vars in 1148 bytes} [Tue May 17 19:10:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:57.727527 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a85543-757f-42ab-8a35-92dcebe7dc5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:57.728031 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14507de-abe5-4707-bc72-e07633b6ae7a None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:57.728211 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a85543-757f-42ab-8a35-92dcebe7dc5d None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:57.728465 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a85543-757f-42ab-8a35-92dcebe7dc5d None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:57.728735 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a85543-757f-42ab-8a35-92dcebe7dc5d None admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:57.729683 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a85543-757f-42ab-8a35-92dcebe7dc5d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:57.729985 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a85543-757f-42ab-8a35-92dcebe7dc5d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:57.731473 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a85543-757f-42ab-8a35-92dcebe7dc5d None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:57.758699 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 344/679] 10.222.0.58 () {56 vars in 1135 bytes} [Tue May 17 19:10:57 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:57.760042 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a85543-757f-42ab-8a35-92dcebe7dc5d None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:57.851346 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14507de-abe5-4707-bc72-e07633b6ae7a None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:57.858121 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a85543-757f-42ab-8a35-92dcebe7dc5d None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:57.895833 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c99dc4-fdb2-4847-8cd0-264b9f958c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:57.896056 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50d5a1f-6b49-4b72-a3d7-3fbde1b2a1d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:57.896491 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50d5a1f-6b49-4b72-a3d7-3fbde1b2a1d0 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:57.896765 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50d5a1f-6b49-4b72-a3d7-3fbde1b2a1d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:57.897034 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50d5a1f-6b49-4b72-a3d7-3fbde1b2a1d0 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:57.897619 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c99dc4-fdb2-4847-8cd0-264b9f958c1b None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:57.897722 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50d5a1f-6b49-4b72-a3d7-3fbde1b2a1d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:57.898023 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50d5a1f-6b49-4b72-a3d7-3fbde1b2a1d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:57.898390 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c99dc4-fdb2-4847-8cd0-264b9f958c1b None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:57.898860 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c99dc4-fdb2-4847-8cd0-264b9f958c1b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:57.899438 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50d5a1f-6b49-4b72-a3d7-3fbde1b2a1d0 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:57.900112 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c99dc4-fdb2-4847-8cd0-264b9f958c1b None admin] RBAC: Adding query filter params (role.id=1754a95b778c4e20b53b20cb52ada92b, scope.project.id=1a9d3a0631284c9fb113266036993e2d, user.id=d05409b38ae346458abfe41ebda1c59e) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:57.900639 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c99dc4-fdb2-4847-8cd0-264b9f958c1b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:57.902244 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c99dc4-fdb2-4847-8cd0-264b9f958c1b None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:57.908704 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 345/680] 10.222.0.58 () {56 vars in 1374 bytes} [Tue May 17 19:10:57 2022] GET /identity/v3/role_assignments?user.id=d05409b38ae346458abfe41ebda1c59e&scope.project.id=1a9d3a0631284c9fb113266036993e2d&role.id=1754a95b778c4e20b53b20cb52ada92b => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:57.909713 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c99dc4-fdb2-4847-8cd0-264b9f958c1b None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:57.911848 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 336/681] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 17 19:10:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:57.920732 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50d5a1f-6b49-4b72-a3d7-3fbde1b2a1d0 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:58.132310 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c99dc4-fdb2-4847-8cd0-264b9f958c1b None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:58.136262 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50d5a1f-6b49-4b72-a3d7-3fbde1b2a1d0 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:58.178216 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaf99e6-bf33-4557-8b67-720611891a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:58.178635 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ea0d99-6dc3-467b-a318-d09feefec0a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:58.179430 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ea0d99-6dc3-467b-a318-d09feefec0a9 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:58.179598 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaf99e6-bf33-4557-8b67-720611891a7f None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:58.179733 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ea0d99-6dc3-467b-a318-d09feefec0a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:58.180015 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ea0d99-6dc3-467b-a318-d09feefec0a9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:58.180282 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaf99e6-bf33-4557-8b67-720611891a7f None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:58.180708 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaf99e6-bf33-4557-8b67-720611891a7f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:58.188101 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ea0d99-6dc3-467b-a318-d09feefec0a9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:58.189208 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ea0d99-6dc3-467b-a318-d09feefec0a9 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:58.203911 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-40ea0d99-6dc3-467b-a318-d09feefec0a9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 17 19:10:58.203911 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:58.203911 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:58.203911 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:58.203911 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:58.203911 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:58.203911 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:58.203911 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:58.203911 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:58.203911 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:58.203911 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:58.203911 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:58.203911 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 17 19:10:58.203911 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(project_id) May 17 19:10:58.203911 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 17 19:10:58.204896 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 17 19:10:58.204896 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:58.204896 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:58.204896 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:58.204896 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:58.204896 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:58.204896 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:58.204896 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:58.204896 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:10:58.204896 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:58.204896 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:58.204896 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:58.204896 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:58.204896 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:58.204896 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:58.204896 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:58.204896 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:58.205770 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 17 19:10:58.205770 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 17 19:10:58.205770 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 17 19:10:58.205770 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 17 19:10:58.205770 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 17 19:10:58.205770 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 17 19:10:58.205770 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 17 19:10:58.205770 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:10:58.205770 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 337/682] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 17 19:10:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:10:58.206633 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f133b67-d2b2-43f8-b6af-791a5b50f1d3 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:58.206923 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f133b67-d2b2-43f8-b6af-791a5b50f1d3 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:58.213554 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f133b67-d2b2-43f8-b6af-791a5b50f1d3 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:58.213964 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 338/683] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:10:58 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:10:58.214713 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f133b67-d2b2-43f8-b6af-791a5b50f1d3 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:58.269115 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcaf99e6-bf33-4557-8b67-720611891a7f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:58.271879 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcaf99e6-bf33-4557-8b67-720611891a7f None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:58.348140 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f133b67-d2b2-43f8-b6af-791a5b50f1d3 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:58.379585 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 346/684] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 17 19:10:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 17 19:10:58.379585 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2fefce-c373-4ffb-91b5-920141f5dd23 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:58.379585 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2fefce-c373-4ffb-91b5-920141f5dd23 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:58.379585 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2fefce-c373-4ffb-91b5-920141f5dd23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:58.390723 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89de6107-9fcf-43e5-bd9e-35b97c4b2708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:58.391615 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89de6107-9fcf-43e5-bd9e-35b97c4b2708 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:58.392094 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89de6107-9fcf-43e5-bd9e-35b97c4b2708 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:58.392536 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89de6107-9fcf-43e5-bd9e-35b97c4b2708 None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:58.393479 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89de6107-9fcf-43e5-bd9e-35b97c4b2708 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:58.393955 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89de6107-9fcf-43e5-bd9e-35b97c4b2708 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:58.395749 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89de6107-9fcf-43e5-bd9e-35b97c4b2708 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:58.395874 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-ec2fefce-c373-4ffb-91b5-920141f5dd23 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:10:58.397057 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-ec2fefce-c373-4ffb-91b5-920141f5dd23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:10:58.408363 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 339/685] 10.222.0.58 () {56 vars in 1148 bytes} [Tue May 17 19:10:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:58.409996 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89de6107-9fcf-43e5-bd9e-35b97c4b2708 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:58.469277 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 347/686] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:10:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 17 19:10:58.470309 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2fefce-c373-4ffb-91b5-920141f5dd23 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:58.514955 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89de6107-9fcf-43e5-bd9e-35b97c4b2708 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:58.550522 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd666fd-46d7-4905-96bb-a5990d08ef8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:58.552040 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd666fd-46d7-4905-96bb-a5990d08ef8b None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:58.552758 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd666fd-46d7-4905-96bb-a5990d08ef8b None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:58.553237 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd666fd-46d7-4905-96bb-a5990d08ef8b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:58.568134 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd666fd-46d7-4905-96bb-a5990d08ef8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:58.569331 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2fefce-c373-4ffb-91b5-920141f5dd23 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:58.570033 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd666fd-46d7-4905-96bb-a5990d08ef8b None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:58.591289 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-7fd666fd-46d7-4905-96bb-a5990d08ef8b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:10:58.591289 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:58.591289 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:58.591289 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:58.591289 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:58.591289 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:58.591289 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:58.591289 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:58.591289 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:58.591289 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:58.591289 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:58.591289 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:58.591289 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 17 19:10:58.591289 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 17 19:10:58.591289 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 17 19:10:58.592279 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 17 19:10:58.592279 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:58.592279 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:58.592279 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:58.592279 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:58.592279 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:58.592279 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:58.592279 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:58.592279 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:10:58.592279 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:58.592279 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:58.592279 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:58.592279 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:58.592279 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:58.592279 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:58.592279 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:58.592279 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:58.593175 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 17 19:10:58.593175 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 17 19:10:58.593175 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:10:58.593175 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:10:58.600921 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 340/687] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:10:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:10:58.600921 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd666fd-46d7-4905-96bb-a5990d08ef8b None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:58.613440 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1fbc72-8b3a-4555-8ecb-c33d59a1ce10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:58.614099 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1fbc72-8b3a-4555-8ecb-c33d59a1ce10 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:58.614377 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1fbc72-8b3a-4555-8ecb-c33d59a1ce10 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:58.614649 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1fbc72-8b3a-4555-8ecb-c33d59a1ce10 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:58.615806 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1fbc72-8b3a-4555-8ecb-c33d59a1ce10 None admin] RBAC: Adding query filter params (role.id=1754a95b778c4e20b53b20cb52ada92b, scope.project.id=1a9d3a0631284c9fb113266036993e2d, user.id=1ac193d15f48463c98ff1d97770ed794) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:58.615806 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1fbc72-8b3a-4555-8ecb-c33d59a1ce10 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:58.617434 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1fbc72-8b3a-4555-8ecb-c33d59a1ce10 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:58.624015 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 348/688] 10.222.0.58 () {56 vars in 1374 bytes} [Tue May 17 19:10:58 2022] GET /identity/v3/role_assignments?user.id=1ac193d15f48463c98ff1d97770ed794&scope.project.id=1a9d3a0631284c9fb113266036993e2d&role.id=1754a95b778c4e20b53b20cb52ada92b => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:58.624808 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1fbc72-8b3a-4555-8ecb-c33d59a1ce10 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:58.721162 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd666fd-46d7-4905-96bb-a5990d08ef8b None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:58.762086 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b49402-72a5-43db-8c81-c8f99bd52920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:58.763250 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b49402-72a5-43db-8c81-c8f99bd52920 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:58.763782 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b49402-72a5-43db-8c81-c8f99bd52920 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:58.764248 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b49402-72a5-43db-8c81-c8f99bd52920 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:58.777270 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b49402-72a5-43db-8c81-c8f99bd52920 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 17 19:10:58.786578 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-f7b49402-72a5-43db-8c81-c8f99bd52920 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 17 19:10:58.786578 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:58.786578 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:58.786578 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:58.786578 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:58.786578 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:58.786578 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:58.786578 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:58.786578 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:58.786578 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:58.786578 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:58.786578 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:58.786578 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 17 19:10:58.786578 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) May 17 19:10:58.786578 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 17 19:10:58.786578 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 17 19:10:58.787609 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 17 19:10:58.787609 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 17 19:10:58.787609 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 17 19:10:58.787609 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 17 19:10:58.787609 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:58.787609 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:58.787609 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:58.787609 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:58.787609 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:58.787609 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:58.787609 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:58.787609 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:10:58.787609 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:58.787609 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:58.787609 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:58.787609 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:58.788451 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:58.788451 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:58.788451 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:58.788451 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:58.788451 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 17 19:10:58.788451 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 17 19:10:58.788451 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 17 19:10:58.788451 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 17 19:10:58.788451 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 17 19:10:58.788451 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 17 19:10:58.788451 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 17 19:10:58.788451 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:10:58.791939 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 341/689] 10.222.0.58 () {56 vars in 1096 bytes} [Tue May 17 19:10:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:10:58.798305 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b49402-72a5-43db-8c81-c8f99bd52920 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:58.833145 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1fbc72-8b3a-4555-8ecb-c33d59a1ce10 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:58.874499 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44adbc3-38e0-4cef-b8e1-f8f549db8543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:58.875739 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44adbc3-38e0-4cef-b8e1-f8f549db8543 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:58.876238 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44adbc3-38e0-4cef-b8e1-f8f549db8543 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:58.876707 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44adbc3-38e0-4cef-b8e1-f8f549db8543 None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:58.877633 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44adbc3-38e0-4cef-b8e1-f8f549db8543 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:58.878152 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44adbc3-38e0-4cef-b8e1-f8f549db8543 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:58.879957 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44adbc3-38e0-4cef-b8e1-f8f549db8543 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:58.893073 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 349/690] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 17 19:10:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:58.899130 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44adbc3-38e0-4cef-b8e1-f8f549db8543 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:58.980392 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b49402-72a5-43db-8c81-c8f99bd52920 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:59.016988 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301ffe93-ff03-46b2-88a5-b46ae0303b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:59.017667 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301ffe93-ff03-46b2-88a5-b46ae0303b58 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:59.017945 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301ffe93-ff03-46b2-88a5-b46ae0303b58 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:59.018216 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301ffe93-ff03-46b2-88a5-b46ae0303b58 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:59.018963 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301ffe93-ff03-46b2-88a5-b46ae0303b58 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:59.019263 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301ffe93-ff03-46b2-88a5-b46ae0303b58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:59.020800 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301ffe93-ff03-46b2-88a5-b46ae0303b58 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:59.029212 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44adbc3-38e0-4cef-b8e1-f8f549db8543 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:59.029726 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 342/691] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 17 19:10:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:59.036739 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301ffe93-ff03-46b2-88a5-b46ae0303b58 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:59.067851 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04f49b1-52ca-47a9-8abc-9d8c469f68f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:59.068653 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04f49b1-52ca-47a9-8abc-9d8c469f68f8 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:59.068927 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04f49b1-52ca-47a9-8abc-9d8c469f68f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:59.069201 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04f49b1-52ca-47a9-8abc-9d8c469f68f8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:59.077101 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04f49b1-52ca-47a9-8abc-9d8c469f68f8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:59.078552 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04f49b1-52ca-47a9-8abc-9d8c469f68f8 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:59.093209 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-f04f49b1-52ca-47a9-8abc-9d8c469f68f8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 17 19:10:59.093209 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:59.093209 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:59.093209 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:59.093209 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:59.093209 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:59.093209 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:59.093209 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:59.093209 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:59.093209 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:59.093209 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:59.093209 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:59.093209 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 17 19:10:59.093209 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(project_id) May 17 19:10:59.093209 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 17 19:10:59.094142 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 17 19:10:59.094142 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:59.094142 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:59.094142 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:59.094142 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:59.094142 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:59.094142 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:59.094142 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:59.094142 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:10:59.094142 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:59.094142 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:59.094142 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:59.094142 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:59.094142 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:59.094142 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:59.094142 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:59.094142 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:59.095072 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 17 19:10:59.095072 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 17 19:10:59.095072 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 17 19:10:59.095072 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 17 19:10:59.095072 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 17 19:10:59.095072 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 17 19:10:59.095072 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 17 19:10:59.095072 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:10:59.095072 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 350/692] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 17 19:10:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:10:59.100459 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04f49b1-52ca-47a9-8abc-9d8c469f68f8 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:59.164897 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301ffe93-ff03-46b2-88a5-b46ae0303b58 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:59.216166 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8f8268-c231-4655-bb51-fde223eb6dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:59.217371 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8f8268-c231-4655-bb51-fde223eb6dc3 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:59.217843 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8f8268-c231-4655-bb51-fde223eb6dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:59.218307 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8f8268-c231-4655-bb51-fde223eb6dc3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:59.236758 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8f8268-c231-4655-bb51-fde223eb6dc3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:59.238989 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8f8268-c231-4655-bb51-fde223eb6dc3 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:59.252965 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04f49b1-52ca-47a9-8abc-9d8c469f68f8 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:59.259990 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-ef8f8268-c231-4655-bb51-fde223eb6dc3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:10:59.259990 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:59.259990 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:59.259990 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:59.259990 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:59.259990 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:59.259990 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:59.259990 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:59.259990 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:59.259990 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:59.259990 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:59.259990 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:59.259990 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 17 19:10:59.259990 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 17 19:10:59.259990 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 17 19:10:59.260919 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 17 19:10:59.260919 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:59.260919 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:59.260919 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:59.260919 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:59.260919 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:59.260919 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:59.260919 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:59.260919 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:10:59.260919 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:59.260919 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:59.260919 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:59.260919 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:59.260919 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:59.260919 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:59.260919 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:59.260919 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:59.261855 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 17 19:10:59.261855 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 17 19:10:59.261855 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:10:59.261855 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:10:59.265904 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 343/693] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:10:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:10:59.267465 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc05a30-feb9-41e7-baed-8bab63e2a1ac None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:59.267814 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc05a30-feb9-41e7-baed-8bab63e2a1ac None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:59.268617 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc05a30-feb9-41e7-baed-8bab63e2a1ac None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:59.274751 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 344/694] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:10:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:10:59.275745 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc05a30-feb9-41e7-baed-8bab63e2a1ac None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:59.298967 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edcbf84-37b3-489c-97a5-c1465d3dd5d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:59.299656 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edcbf84-37b3-489c-97a5-c1465d3dd5d4 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:59.299934 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edcbf84-37b3-489c-97a5-c1465d3dd5d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:59.300212 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edcbf84-37b3-489c-97a5-c1465d3dd5d4 None admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:59.301065 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edcbf84-37b3-489c-97a5-c1465d3dd5d4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:59.301621 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edcbf84-37b3-489c-97a5-c1465d3dd5d4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:59.302765 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edcbf84-37b3-489c-97a5-c1465d3dd5d4 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:59.314757 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 351/695] 10.222.0.58 () {56 vars in 1148 bytes} [Tue May 17 19:10:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:59.316150 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5352ccd3-8768-4991-997e-33b4a8287251 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:59.316474 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5352ccd3-8768-4991-997e-33b4a8287251 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:59.316749 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5352ccd3-8768-4991-997e-33b4a8287251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:59.333634 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-5352ccd3-8768-4991-997e-33b4a8287251 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:10:59.334906 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-5352ccd3-8768-4991-997e-33b4a8287251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:10:59.391716 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc05a30-feb9-41e7-baed-8bab63e2a1ac None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:59.411605 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 352/696] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:10:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 17 19:10:59.412550 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5352ccd3-8768-4991-997e-33b4a8287251 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:59.427014 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d862c5-5d35-4274-9629-e794e3fd7867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:59.427716 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d862c5-5d35-4274-9629-e794e3fd7867 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:59.427997 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d862c5-5d35-4274-9629-e794e3fd7867 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:59.428269 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d862c5-5d35-4274-9629-e794e3fd7867 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:59.429097 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d862c5-5d35-4274-9629-e794e3fd7867 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:59.429400 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d862c5-5d35-4274-9629-e794e3fd7867 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:59.430770 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d862c5-5d35-4274-9629-e794e3fd7867 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:59.443872 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 345/697] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 17 19:10:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:59.449995 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d862c5-5d35-4274-9629-e794e3fd7867 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:59.531111 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5352ccd3-8768-4991-997e-33b4a8287251 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:59.565047 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78a1535-8108-4c9b-8f4d-a5addddc5313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:59.565697 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78a1535-8108-4c9b-8f4d-a5addddc5313 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:59.565978 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78a1535-8108-4c9b-8f4d-a5addddc5313 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:59.567207 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78a1535-8108-4c9b-8f4d-a5addddc5313 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:59.567789 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78a1535-8108-4c9b-8f4d-a5addddc5313 None admin] RBAC: Adding query filter params (role.id=1754a95b778c4e20b53b20cb52ada92b, scope.project.id=1a9d3a0631284c9fb113266036993e2d, user.id=b0cb292002434d30bc81f370d120a2a2) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:59.567989 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78a1535-8108-4c9b-8f4d-a5addddc5313 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:59.569438 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78a1535-8108-4c9b-8f4d-a5addddc5313 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:59.572640 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d862c5-5d35-4274-9629-e794e3fd7867 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:59.575357 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 353/698] 10.222.0.58 () {56 vars in 1374 bytes} [Tue May 17 19:10:59 2022] GET /identity/v3/role_assignments?user.id=b0cb292002434d30bc81f370d120a2a2&scope.project.id=1a9d3a0631284c9fb113266036993e2d&role.id=1754a95b778c4e20b53b20cb52ada92b => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:10:59.576789 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78a1535-8108-4c9b-8f4d-a5addddc5313 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:59.616620 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bdf8e8-f275-4e84-9cb2-03d01a5c0e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:59.617759 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bdf8e8-f275-4e84-9cb2-03d01a5c0e4a None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:59.618081 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bdf8e8-f275-4e84-9cb2-03d01a5c0e4a None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:59.618357 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bdf8e8-f275-4e84-9cb2-03d01a5c0e4a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:59.627463 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bdf8e8-f275-4e84-9cb2-03d01a5c0e4a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:59.629901 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bdf8e8-f275-4e84-9cb2-03d01a5c0e4a None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:59.648247 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-06bdf8e8-f275-4e84-9cb2-03d01a5c0e4a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 17 19:10:59.648247 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:59.648247 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:59.648247 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:59.648247 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:59.648247 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:59.648247 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:59.648247 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:59.648247 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:59.648247 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:59.648247 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:59.648247 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:59.648247 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 17 19:10:59.648247 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_user(user_id) May 17 19:10:59.648247 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 17 19:10:59.648247 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 17 19:10:59.649291 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:59.649291 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:59.649291 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 17 19:10:59.649291 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 17 19:10:59.649291 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 17 19:10:59.649291 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 17 19:10:59.649291 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:59.649291 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:59.649291 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:59.649291 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:59.649291 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:59.649291 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:10:59.649291 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:59.649291 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:59.649291 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:59.649291 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:59.649291 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:59.649291 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:59.650338 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:59.650338 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:59.650338 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 17 19:10:59.650338 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 17 19:10:59.650338 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 17 19:10:59.650338 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 17 19:10:59.650338 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 17 19:10:59.650338 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 17 19:10:59.650338 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 17 19:10:59.650338 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:10:59.650338 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 346/699] 10.222.0.58 () {56 vars in 1093 bytes} [Tue May 17 19:10:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:10:59.658090 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bdf8e8-f275-4e84-9cb2-03d01a5c0e4a None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:59.790600 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78a1535-8108-4c9b-8f4d-a5addddc5313 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:59.841464 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bdf8e8-f275-4e84-9cb2-03d01a5c0e4a None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:10:59.852200 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a41ba85-6517-4865-a076-a7869e639a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:59.853388 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a41ba85-6517-4865-a076-a7869e639a06 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:59.853874 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a41ba85-6517-4865-a076-a7869e639a06 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:59.854345 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a41ba85-6517-4865-a076-a7869e639a06 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:59.874008 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a41ba85-6517-4865-a076-a7869e639a06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:59.878999 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a41ba85-6517-4865-a076-a7869e639a06 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:59.881483 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8474f830-9533-4bbd-a879-7ae77afc5763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:10:59.882159 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8474f830-9533-4bbd-a879-7ae77afc5763 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:10:59.882434 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8474f830-9533-4bbd-a879-7ae77afc5763 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:10:59.882705 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8474f830-9533-4bbd-a879-7ae77afc5763 None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:10:59.883573 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8474f830-9533-4bbd-a879-7ae77afc5763 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:10:59.883884 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8474f830-9533-4bbd-a879-7ae77afc5763 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:10:59.885315 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8474f830-9533-4bbd-a879-7ae77afc5763 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:10:59.900100 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-0a41ba85-6517-4865-a076-a7869e639a06 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:10:59.900100 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:10:59.900100 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:10:59.900100 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:10:59.900100 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:10:59.900100 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:10:59.900100 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:10:59.900100 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:10:59.900100 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:10:59.900100 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:10:59.900100 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:10:59.900100 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:10:59.900100 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 17 19:10:59.900100 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 17 19:10:59.900100 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 17 19:10:59.901062 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 17 19:10:59.901062 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:10:59.901062 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:10:59.901062 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:10:59.901062 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:10:59.901062 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:10:59.901062 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:10:59.901062 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:10:59.901062 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:10:59.901062 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:10:59.901062 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:10:59.901062 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:10:59.901062 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:10:59.901062 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:10:59.901062 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:10:59.901062 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:10:59.901062 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:10:59.901967 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 17 19:10:59.901967 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 17 19:10:59.901967 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:10:59.901967 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:10:59.904236 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 354/701] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:10:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:10:59.904456 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 347/701] 10.222.0.58 () {56 vars in 1133 bytes} [Tue May 17 19:10:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:10:59.912328 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a41ba85-6517-4865-a076-a7869e639a06 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:10:59.912915 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8474f830-9533-4bbd-a879-7ae77afc5763 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:00.027512 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a41ba85-6517-4865-a076-a7869e639a06 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:00.039855 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8474f830-9533-4bbd-a879-7ae77afc5763 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:00.067521 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66c475e-8ebc-4139-9e41-76ea55e0306f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:00.068879 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66c475e-8ebc-4139-9e41-76ea55e0306f None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:00.069315 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66c475e-8ebc-4139-9e41-76ea55e0306f None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:00.069999 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66c475e-8ebc-4139-9e41-76ea55e0306f None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:00.071026 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66c475e-8ebc-4139-9e41-76ea55e0306f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:00.071500 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66c475e-8ebc-4139-9e41-76ea55e0306f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:00.073449 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66c475e-8ebc-4139-9e41-76ea55e0306f None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:00.077536 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f598277-13be-44cb-bbe0-31b551fb7a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:00.078345 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f598277-13be-44cb-bbe0-31b551fb7a15 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:00.078718 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f598277-13be-44cb-bbe0-31b551fb7a15 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:00.079120 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f598277-13be-44cb-bbe0-31b551fb7a15 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:00.089073 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 355/702] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 17 19:10:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:00.094592 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66c475e-8ebc-4139-9e41-76ea55e0306f None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:00.161044 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f598277-13be-44cb-bbe0-31b551fb7a15 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:00.163190 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f598277-13be-44cb-bbe0-31b551fb7a15 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:00.215440 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66c475e-8ebc-4139-9e41-76ea55e0306f None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:00.264299 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82290367-c569-4a61-8736-5beb2cd8fce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:00.265231 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82290367-c569-4a61-8736-5beb2cd8fce4 None admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:00.265685 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82290367-c569-4a61-8736-5beb2cd8fce4 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:00.266115 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82290367-c569-4a61-8736-5beb2cd8fce4 None admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:00.266969 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82290367-c569-4a61-8736-5beb2cd8fce4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:00.273109 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82290367-c569-4a61-8736-5beb2cd8fce4 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:00.307924 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 348/703] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 17 19:10:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 17 19:11:00.309496 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f598277-13be-44cb-bbe0-31b551fb7a15 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:00.413122 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f598277-13be-44cb-bbe0-31b551fb7a15 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:00.438600 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 356/704] 10.222.0.58 () {60 vars in 1132 bytes} [Tue May 17 19:11:00 2022] POST /identity/v3/users => generated 254 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:11:00.439931 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccd95c2-b74f-4df5-934c-aaf09e645fef None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:00.440212 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccd95c2-b74f-4df5-934c-aaf09e645fef None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:00.447817 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccd95c2-b74f-4df5-934c-aaf09e645fef None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:00.447976 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 357/705] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:11:00 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:11:00.449668 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c078a24-d837-428a-8b6b-8d494a68db09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:00.450675 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c078a24-d837-428a-8b6b-8d494a68db09 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:00.451105 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c078a24-d837-428a-8b6b-8d494a68db09 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:00.451567 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c078a24-d837-428a-8b6b-8d494a68db09 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:00.464081 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c078a24-d837-428a-8b6b-8d494a68db09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:00.465127 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56acb72e-0279-4fda-af7c-8a83709b1e5d None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:00.465411 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56acb72e-0279-4fda-af7c-8a83709b1e5d None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:00.465683 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56acb72e-0279-4fda-af7c-8a83709b1e5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:00.466333 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c078a24-d837-428a-8b6b-8d494a68db09 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:00.485718 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-9c078a24-d837-428a-8b6b-8d494a68db09 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:11:00.485718 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:00.485718 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:11:00.485718 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:11:00.485718 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:11:00.485718 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:11:00.485718 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:11:00.485718 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:11:00.485718 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:11:00.485718 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:11:00.485718 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:11:00.485718 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:11:00.485718 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 17 19:11:00.485718 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 17 19:11:00.485718 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 17 19:11:00.486892 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 17 19:11:00.486892 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:11:00.486892 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:11:00.486892 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:11:00.486892 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:11:00.486892 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:11:00.486892 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:11:00.486892 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:11:00.486892 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:11:00.486892 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:11:00.486892 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:11:00.486892 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:11:00.486892 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:11:00.486892 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:11:00.486892 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:11:00.486892 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:11:00.486892 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:11:00.488089 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 17 19:11:00.488089 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 17 19:11:00.488089 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:11:00.488089 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:11:00.489971 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 349/706] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:11:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 17 19:11:00.495667 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c078a24-d837-428a-8b6b-8d494a68db09 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:00.508653 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-56acb72e-0279-4fda-af7c-8a83709b1e5d None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:11:00.509862 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-56acb72e-0279-4fda-af7c-8a83709b1e5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:11:00.603678 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c078a24-d837-428a-8b6b-8d494a68db09 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:00.654496 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 358/707] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:11:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 17 19:11:00.669019 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f207bbb-73a8-445c-b822-ad98ae6279d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:00.669969 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f207bbb-73a8-445c-b822-ad98ae6279d3 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:00.670398 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f207bbb-73a8-445c-b822-ad98ae6279d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:00.670832 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f207bbb-73a8-445c-b822-ad98ae6279d3 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:00.671754 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f207bbb-73a8-445c-b822-ad98ae6279d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:00.672224 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f207bbb-73a8-445c-b822-ad98ae6279d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:00.673753 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f207bbb-73a8-445c-b822-ad98ae6279d3 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:00.684566 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56acb72e-0279-4fda-af7c-8a83709b1e5d None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:00.694392 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 350/708] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 17 19:11:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:11:00.695579 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f207bbb-73a8-445c-b822-ad98ae6279d3 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:00.806833 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f207bbb-73a8-445c-b822-ad98ae6279d3 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:00.813254 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56acb72e-0279-4fda-af7c-8a83709b1e5d None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:00.848665 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f5a982-68de-43fa-bda0-448692ae615f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:00.849041 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e3bdfe-3baa-4b30-8418-3e534b93fdbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:00.849457 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f5a982-68de-43fa-bda0-448692ae615f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:00.849730 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f5a982-68de-43fa-bda0-448692ae615f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:00.849995 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f5a982-68de-43fa-bda0-448692ae615f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:00.850788 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e3bdfe-3baa-4b30-8418-3e534b93fdbc None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:00.851246 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e3bdfe-3baa-4b30-8418-3e534b93fdbc None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:00.851730 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e3bdfe-3baa-4b30-8418-3e534b93fdbc None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:00.852561 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e3bdfe-3baa-4b30-8418-3e534b93fdbc None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:00.854125 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e3bdfe-3baa-4b30-8418-3e534b93fdbc None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:00.857890 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f5a982-68de-43fa-bda0-448692ae615f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:00.858773 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f5a982-68de-43fa-bda0-448692ae615f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:00.874277 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-50f5a982-68de-43fa-bda0-448692ae615f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 17 19:11:00.874277 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:00.874277 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:11:00.874277 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:11:00.874277 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:11:00.874277 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:11:00.874277 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:11:00.874277 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:11:00.874277 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:11:00.874277 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:11:00.874277 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:11:00.874277 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:11:00.874277 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 17 19:11:00.874277 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(project_id) May 17 19:11:00.874277 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 17 19:11:00.875224 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 17 19:11:00.875224 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:11:00.875224 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:11:00.875224 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:11:00.875224 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:11:00.875224 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:11:00.875224 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:11:00.875224 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:11:00.875224 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:11:00.875224 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:11:00.875224 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:11:00.875224 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:11:00.875224 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:11:00.875224 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:11:00.875224 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:11:00.875224 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:11:00.875224 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:11:00.876246 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 17 19:11:00.876246 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 17 19:11:00.876246 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 17 19:11:00.876246 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 17 19:11:00.876246 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 17 19:11:00.876246 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 17 19:11:00.876246 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 17 19:11:00.876246 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:11:00.878839 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-84e3bdfe-3baa-4b30-8418-3e534b93fdbc None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 17 19:11:00.878839 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:00.878839 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 17 19:11:00.878839 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 17 19:11:00.878839 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 17 19:11:00.878839 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 17 19:11:00.878839 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 17 19:11:00.878839 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 17 19:11:00.878839 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 17 19:11:00.878839 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:11:00.878839 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 17 19:11:00.878839 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:11:00.878839 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:00.878839 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:11:00.878839 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:11:00.878839 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:11:00.879966 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:11:00.879966 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:11:00.879966 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:11:00.879966 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:11:00.879966 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:11:00.879966 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:11:00.879966 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:11:00.879966 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 17 19:11:00.879966 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_service(service_id) May 17 19:11:00.879966 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 17 19:11:00.879966 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 17 19:11:00.879966 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:11:00.879966 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:11:00.879966 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:11:00.879966 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:11:00.879966 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:11:00.880967 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:11:00.880967 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:11:00.880967 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:11:00.880967 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:11:00.880967 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:11:00.880967 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:11:00.880967 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:11:00.880967 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:11:00.880967 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:11:00.880967 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:11:00.880967 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:11:00.880967 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 17 19:11:00.880967 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 17 19:11:00.880967 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 17 19:11:00.880967 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:11:00.881772 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 351/709] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 17 19:11:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 17 19:11:00.881772 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f5a982-68de-43fa-bda0-448692ae615f None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:00.884177 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 359/710] 10.222.0.58 () {56 vars in 1129 bytes} [Tue May 17 19:11:00 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 17 19:11:00.890133 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e3bdfe-3baa-4b30-8418-3e534b93fdbc None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:01.007042 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f5a982-68de-43fa-bda0-448692ae615f None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:01.008766 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e3bdfe-3baa-4b30-8418-3e534b93fdbc None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:01.047313 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77265ab-fb33-42e1-b438-6243df9c0fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:01.047996 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77265ab-fb33-42e1-b438-6243df9c0fb9 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:01.048275 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77265ab-fb33-42e1-b438-6243df9c0fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:01.048546 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77265ab-fb33-42e1-b438-6243df9c0fb9 None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:01.049314 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77265ab-fb33-42e1-b438-6243df9c0fb9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:01.049616 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77265ab-fb33-42e1-b438-6243df9c0fb9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:01.051633 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77265ab-fb33-42e1-b438-6243df9c0fb9 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:01.052701 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcd9cab-9527-4b09-ae48-bf6fdbab6891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:01.053555 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcd9cab-9527-4b09-ae48-bf6fdbab6891 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:01.054014 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcd9cab-9527-4b09-ae48-bf6fdbab6891 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:01.054465 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcd9cab-9527-4b09-ae48-bf6fdbab6891 None admin] PATH_INFO: `/v3/services` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:01.055346 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcd9cab-9527-4b09-ae48-bf6fdbab6891 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:01.055870 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcd9cab-9527-4b09-ae48-bf6fdbab6891 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:01.057407 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcd9cab-9527-4b09-ae48-bf6fdbab6891 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:01.062317 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 360/711] 10.222.0.58 () {56 vars in 1124 bytes} [Tue May 17 19:11:00 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:11:01.067273 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 352/712] 10.222.0.58 () {56 vars in 1148 bytes} [Tue May 17 19:11:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:01.068376 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcd9cab-9527-4b09-ae48-bf6fdbab6891 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:01.070338 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77265ab-fb33-42e1-b438-6243df9c0fb9 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:01.175857 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcd9cab-9527-4b09-ae48-bf6fdbab6891 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:01.232510 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71f5c0b-77d4-43a2-b834-10d313653714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:01.233131 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71f5c0b-77d4-43a2-b834-10d313653714 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:01.233411 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71f5c0b-77d4-43a2-b834-10d313653714 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:01.233684 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71f5c0b-77d4-43a2-b834-10d313653714 None admin] PATH_INFO: `/v3/services` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:01.234409 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71f5c0b-77d4-43a2-b834-10d313653714 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:01.234711 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71f5c0b-77d4-43a2-b834-10d313653714 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:01.236127 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71f5c0b-77d4-43a2-b834-10d313653714 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:01.242808 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 361/713] 10.222.0.58 () {56 vars in 1124 bytes} [Tue May 17 19:11:01 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:01.364440 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77265ab-fb33-42e1-b438-6243df9c0fb9 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:01.414207 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36306bb5-5254-4270-b942-9040c49c16db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:01.415000 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36306bb5-5254-4270-b942-9040c49c16db None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:01.415168 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36306bb5-5254-4270-b942-9040c49c16db None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:01.415405 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36306bb5-5254-4270-b942-9040c49c16db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:01.416270 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36306bb5-5254-4270-b942-9040c49c16db None admin] RBAC: Adding query filter params (role.id=60af5088793e4379b88821a86552a344, scope.project.id=1a9d3a0631284c9fb113266036993e2d, user.id=65c1f878b0dc4d63a4f5dcb12fac1dc1) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:01.416575 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36306bb5-5254-4270-b942-9040c49c16db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:01.418020 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36306bb5-5254-4270-b942-9040c49c16db None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:01.428849 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 353/714] 10.222.0.58 () {56 vars in 1374 bytes} [Tue May 17 19:11:01 2022] GET /identity/v3/role_assignments?user.id=65c1f878b0dc4d63a4f5dcb12fac1dc1&scope.project.id=1a9d3a0631284c9fb113266036993e2d&role.id=60af5088793e4379b88821a86552a344 => generated 259 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:01.534031 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af810d9e-d65d-4be5-930d-04f8ab42f9d6 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:01.534644 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af810d9e-d65d-4be5-930d-04f8ab42f9d6 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:01.535214 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af810d9e-d65d-4be5-930d-04f8ab42f9d6 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:01.536938 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 362/715] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:11:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:11:01.543165 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005c8ffc-068c-4842-bc58-dbf5a9eb9866 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:01.543472 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005c8ffc-068c-4842-bc58-dbf5a9eb9866 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:01.543746 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005c8ffc-068c-4842-bc58-dbf5a9eb9866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:01.561623 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-005c8ffc-068c-4842-bc58-dbf5a9eb9866 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:11:01.563221 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-005c8ffc-068c-4842-bc58-dbf5a9eb9866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:11:01.638315 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 354/716] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:11:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 17 19:11:01.664295 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af810d9e-d65d-4be5-930d-04f8ab42f9d6 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:01.769008 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af810d9e-d65d-4be5-930d-04f8ab42f9d6 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:01.807745 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87234b03-0adc-454a-b082-17534474130a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:01.809071 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87234b03-0adc-454a-b082-17534474130a None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:01.809567 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87234b03-0adc-454a-b082-17534474130a None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:01.810051 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87234b03-0adc-454a-b082-17534474130a None admin] PATH_INFO: `/v3/services/network` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:01.810977 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87234b03-0adc-454a-b082-17534474130a None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:01.812686 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87234b03-0adc-454a-b082-17534474130a None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:01.825770 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-87234b03-0adc-454a-b082-17534474130a None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 17 19:11:01.825770 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:01.825770 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 17 19:11:01.825770 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 17 19:11:01.825770 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 17 19:11:01.825770 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 17 19:11:01.825770 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 17 19:11:01.825770 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 17 19:11:01.825770 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 17 19:11:01.825770 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:11:01.825770 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 17 19:11:01.825770 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:11:01.825770 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:01.825770 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:11:01.825770 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:11:01.825770 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:11:01.825770 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:11:01.826803 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:11:01.826803 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:11:01.826803 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:11:01.826803 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:11:01.826803 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:11:01.826803 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:11:01.826803 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 17 19:11:01.826803 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_service(service_id) May 17 19:11:01.826803 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 17 19:11:01.826803 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 17 19:11:01.826803 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:11:01.826803 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:11:01.826803 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:11:01.826803 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:11:01.826803 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:11:01.826803 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:11:01.827698 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:11:01.827698 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:11:01.827698 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:11:01.827698 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:11:01.827698 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:11:01.827698 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:11:01.827698 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:11:01.827698 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:11:01.827698 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:11:01.827698 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:11:01.827698 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 17 19:11:01.827698 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 17 19:11:01.827698 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 17 19:11:01.827698 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:11:01.829949 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 363/717] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 17 19:11:01 2022] GET /identity/v3/services/network => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:11:01.835087 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005c8ffc-068c-4842-bc58-dbf5a9eb9866 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:02.004539 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005c8ffc-068c-4842-bc58-dbf5a9eb9866 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:02.055056 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82edb56-edff-445f-8b32-0f513b9760f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:02.055787 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82edb56-edff-445f-8b32-0f513b9760f5 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:02.056060 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82edb56-edff-445f-8b32-0f513b9760f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:02.056326 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82edb56-edff-445f-8b32-0f513b9760f5 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:02.057046 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82edb56-edff-445f-8b32-0f513b9760f5 None admin] RBAC: Adding query filter params (name=network) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:02.057344 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82edb56-edff-445f-8b32-0f513b9760f5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:02.058825 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82edb56-edff-445f-8b32-0f513b9760f5 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:02.064323 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 355/718] 10.222.0.58 () {56 vars in 1112 bytes} [Tue May 17 19:11:01 2022] GET /identity/v3/services?name=network => generated 124 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:02.069842 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87234b03-0adc-454a-b082-17534474130a None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:02.190595 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87234b03-0adc-454a-b082-17534474130a None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:02.230922 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f5d9fa-a2f5-438a-a054-c1d8f8c29c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:02.231610 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f5d9fa-a2f5-438a-a054-c1d8f8c29c5e None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:02.231887 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f5d9fa-a2f5-438a-a054-c1d8f8c29c5e None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:02.232157 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f5d9fa-a2f5-438a-a054-c1d8f8c29c5e None admin] PATH_INFO: `/v3/services` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:02.232862 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f5d9fa-a2f5-438a-a054-c1d8f8c29c5e None admin] RBAC: Adding query filter params (type=network) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:02.233162 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f5d9fa-a2f5-438a-a054-c1d8f8c29c5e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:02.234525 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f5d9fa-a2f5-438a-a054-c1d8f8c29c5e None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:02.239231 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 364/719] 10.222.0.58 () {56 vars in 1112 bytes} [Tue May 17 19:11:02 2022] GET /identity/v3/services?type=network => generated 124 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:02.598999 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fe799a-fb99-4feb-b2a5-31011a23c961 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:02.600050 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fe799a-fb99-4feb-b2a5-31011a23c961 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:02.600550 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fe799a-fb99-4feb-b2a5-31011a23c961 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:02.602384 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 356/720] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:11:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:11:02.607780 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f156e9-5fac-43e5-8a8b-c69e6439255d None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:02.608070 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f156e9-5fac-43e5-8a8b-c69e6439255d None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:02.608348 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f156e9-5fac-43e5-8a8b-c69e6439255d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:02.627587 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-a6f156e9-5fac-43e5-8a8b-c69e6439255d None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:11:02.629289 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-a6f156e9-5fac-43e5-8a8b-c69e6439255d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:11:02.713023 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 365/721] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:11:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 17 19:11:02.749497 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fe799a-fb99-4feb-b2a5-31011a23c961 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:02.885189 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fe799a-fb99-4feb-b2a5-31011a23c961 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:02.933114 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6933c5a-1ff0-42b9-bc9c-8613c499a86a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:02.933915 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6933c5a-1ff0-42b9-bc9c-8613c499a86a None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:02.934191 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6933c5a-1ff0-42b9-bc9c-8613c499a86a None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:02.934457 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6933c5a-1ff0-42b9-bc9c-8613c499a86a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:02.949819 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6933c5a-1ff0-42b9-bc9c-8613c499a86a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 17 19:11:02.959864 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-e6933c5a-1ff0-42b9-bc9c-8613c499a86a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 17 19:11:02.959864 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:02.959864 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:11:02.959864 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:11:02.959864 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:11:02.959864 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:11:02.959864 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:11:02.959864 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:11:02.959864 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:11:02.959864 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:11:02.959864 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:11:02.959864 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:11:02.959864 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 17 19:11:02.959864 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) May 17 19:11:02.959864 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 17 19:11:02.959864 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 17 19:11:02.961153 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 17 19:11:02.961153 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 17 19:11:02.961153 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 17 19:11:02.961153 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 17 19:11:02.961153 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:11:02.961153 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:11:02.961153 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:11:02.961153 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:11:02.961153 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:11:02.961153 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:11:02.961153 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:11:02.961153 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:11:02.961153 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:11:02.961153 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:11:02.961153 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:11:02.961153 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:11:02.962091 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:11:02.962091 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:11:02.962091 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:11:02.962091 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:11:02.962091 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 17 19:11:02.962091 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 17 19:11:02.962091 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 17 19:11:02.962091 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 17 19:11:02.962091 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 17 19:11:02.962091 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 17 19:11:02.962091 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 17 19:11:02.962091 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:11:02.962091 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 357/722] 10.222.0.58 () {56 vars in 1096 bytes} [Tue May 17 19:11:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:11:02.970217 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f156e9-5fac-43e5-8a8b-c69e6439255d None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:03.117227 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f156e9-5fac-43e5-8a8b-c69e6439255d None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:03.158747 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb6fcc2-c05e-43e8-a091-42fe55f102e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:03.160477 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb6fcc2-c05e-43e8-a091-42fe55f102e1 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:03.160953 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb6fcc2-c05e-43e8-a091-42fe55f102e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:03.161435 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb6fcc2-c05e-43e8-a091-42fe55f102e1 None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:03.162355 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb6fcc2-c05e-43e8-a091-42fe55f102e1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:03.162879 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb6fcc2-c05e-43e8-a091-42fe55f102e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:03.164469 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb6fcc2-c05e-43e8-a091-42fe55f102e1 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:03.177728 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 366/723] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 17 19:11:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:11:03.184544 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6933c5a-1ff0-42b9-bc9c-8613c499a86a None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:03.328721 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6933c5a-1ff0-42b9-bc9c-8613c499a86a None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:03.378616 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3e7f1b-75fe-4690-a261-5ae23b3d6b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:03.379417 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3e7f1b-75fe-4690-a261-5ae23b3d6b54 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:03.379722 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3e7f1b-75fe-4690-a261-5ae23b3d6b54 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:03.379998 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3e7f1b-75fe-4690-a261-5ae23b3d6b54 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:03.393395 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3e7f1b-75fe-4690-a261-5ae23b3d6b54 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:03.394906 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3e7f1b-75fe-4690-a261-5ae23b3d6b54 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:03.414543 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-db3e7f1b-75fe-4690-a261-5ae23b3d6b54 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:11:03.414543 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:03.414543 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:11:03.414543 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:11:03.414543 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:11:03.414543 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:11:03.414543 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:11:03.414543 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:11:03.414543 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:11:03.414543 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:11:03.414543 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:11:03.414543 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:11:03.414543 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 17 19:11:03.414543 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 17 19:11:03.414543 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 17 19:11:03.415463 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 17 19:11:03.415463 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:11:03.415463 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:11:03.415463 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:11:03.415463 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:11:03.415463 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:11:03.415463 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:11:03.415463 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:11:03.415463 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:11:03.415463 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:11:03.415463 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:11:03.415463 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:11:03.415463 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:11:03.415463 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:11:03.415463 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:11:03.415463 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:11:03.415463 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:11:03.416748 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 17 19:11:03.416748 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 17 19:11:03.416748 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:11:03.416748 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:11:03.420200 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 358/724] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:11:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:11:03.421613 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb6fcc2-c05e-43e8-a091-42fe55f102e1 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:03.461106 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cb736d-07d8-4c6b-b2d2-9826a7aa604a None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:03.461451 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cb736d-07d8-4c6b-b2d2-9826a7aa604a None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:03.461662 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cb736d-07d8-4c6b-b2d2-9826a7aa604a None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:03.463286 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 359/725] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:11:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:11:03.468840 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02e0b94-c268-4acf-825d-170098701682 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:03.469124 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02e0b94-c268-4acf-825d-170098701682 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:03.469396 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02e0b94-c268-4acf-825d-170098701682 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:03.508518 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-a02e0b94-c268-4acf-825d-170098701682 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:11:03.509707 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a02e0b94-c268-4acf-825d-170098701682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:11:03.588413 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb6fcc2-c05e-43e8-a091-42fe55f102e1 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:03.622436 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 360/726] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:11:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 17 19:11:03.648107 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02e0b94-c268-4acf-825d-170098701682 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:03.659074 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd3ef0f-3c1f-4875-9151-8537135990e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:03.660642 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd3ef0f-3c1f-4875-9151-8537135990e2 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:03.662719 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd3ef0f-3c1f-4875-9151-8537135990e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:03.663200 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd3ef0f-3c1f-4875-9151-8537135990e2 None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:03.666473 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd3ef0f-3c1f-4875-9151-8537135990e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:03.667025 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd3ef0f-3c1f-4875-9151-8537135990e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:03.668652 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd3ef0f-3c1f-4875-9151-8537135990e2 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:03.700033 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 367/727] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 17 19:11:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:03.704114 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd3ef0f-3c1f-4875-9151-8537135990e2 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:03.833888 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02e0b94-c268-4acf-825d-170098701682 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:03.898769 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fccfe5-396d-44aa-8570-cd724dba6a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:03.900188 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fccfe5-396d-44aa-8570-cd724dba6a1b None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:03.900297 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd3ef0f-3c1f-4875-9151-8537135990e2 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:03.901099 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fccfe5-396d-44aa-8570-cd724dba6a1b None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:03.901580 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fccfe5-396d-44aa-8570-cd724dba6a1b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:03.918673 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fccfe5-396d-44aa-8570-cd724dba6a1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 17 19:11:03.927103 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-42fccfe5-396d-44aa-8570-cd724dba6a1b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 17 19:11:03.927103 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:03.927103 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:11:03.927103 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:11:03.927103 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:11:03.927103 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:11:03.927103 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:11:03.927103 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:11:03.927103 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:11:03.927103 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:11:03.927103 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:11:03.927103 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:11:03.927103 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 17 19:11:03.927103 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) May 17 19:11:03.927103 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 17 19:11:03.927103 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 17 19:11:03.928370 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 17 19:11:03.928370 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 17 19:11:03.928370 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 17 19:11:03.928370 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 17 19:11:03.928370 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:11:03.928370 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:11:03.928370 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:11:03.928370 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:11:03.928370 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:11:03.928370 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:11:03.928370 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:11:03.928370 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:11:03.928370 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:11:03.928370 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:11:03.928370 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:11:03.928370 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:11:03.929221 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:11:03.929221 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:11:03.929221 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:11:03.929221 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:11:03.929221 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 17 19:11:03.929221 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 17 19:11:03.929221 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 17 19:11:03.929221 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 17 19:11:03.929221 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 17 19:11:03.929221 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 17 19:11:03.929221 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 17 19:11:03.929221 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:11:03.932090 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 361/728] 10.222.0.58 () {56 vars in 1102 bytes} [Tue May 17 19:11:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:11:03.938202 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fccfe5-396d-44aa-8570-cd724dba6a1b None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:03.970690 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a304642-e1c0-4b31-aca4-7c9dd739860e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:03.971768 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a304642-e1c0-4b31-aca4-7c9dd739860e None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:03.972042 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a304642-e1c0-4b31-aca4-7c9dd739860e None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:03.972348 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a304642-e1c0-4b31-aca4-7c9dd739860e None admin] PATH_INFO: `/v3/users/swift` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:03.982490 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a304642-e1c0-4b31-aca4-7c9dd739860e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:03.984086 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a304642-e1c0-4b31-aca4-7c9dd739860e None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:04.006379 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-7a304642-e1c0-4b31-aca4-7c9dd739860e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 17 19:11:04.006379 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:04.006379 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:11:04.006379 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:11:04.006379 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:11:04.006379 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:11:04.006379 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:11:04.006379 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:11:04.006379 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:11:04.006379 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:11:04.006379 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:11:04.006379 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:11:04.006379 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 17 19:11:04.006379 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_user(user_id) May 17 19:11:04.006379 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 17 19:11:04.006379 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 17 19:11:04.007369 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:11:04.007369 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:11:04.007369 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 17 19:11:04.007369 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 17 19:11:04.007369 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 17 19:11:04.007369 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 17 19:11:04.007369 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:11:04.007369 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:11:04.007369 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:11:04.007369 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:11:04.007369 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:11:04.007369 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:11:04.007369 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:11:04.007369 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:11:04.007369 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:11:04.007369 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:11:04.007369 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:11:04.007369 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:11:04.008354 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:11:04.008354 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:11:04.008354 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 17 19:11:04.008354 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 17 19:11:04.008354 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 17 19:11:04.008354 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 17 19:11:04.008354 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 17 19:11:04.008354 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 17 19:11:04.008354 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 17 19:11:04.008354 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:11:04.008354 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 368/729] 10.222.0.58 () {56 vars in 1096 bytes} [Tue May 17 19:11:03 2022] GET /identity/v3/users/swift => generated 83 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:11:04.013175 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a304642-e1c0-4b31-aca4-7c9dd739860e None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:04.053809 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fccfe5-396d-44aa-8570-cd724dba6a1b None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:04.088424 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e792e97f-f285-43f9-93cb-f0dfd15a400c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:04.089434 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e792e97f-f285-43f9-93cb-f0dfd15a400c None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:04.089909 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e792e97f-f285-43f9-93cb-f0dfd15a400c None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:04.090375 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e792e97f-f285-43f9-93cb-f0dfd15a400c None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:04.091397 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e792e97f-f285-43f9-93cb-f0dfd15a400c None admin] RBAC: Adding query filter params (name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:04.092165 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e792e97f-f285-43f9-93cb-f0dfd15a400c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:04.094267 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e792e97f-f285-43f9-93cb-f0dfd15a400c None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:04.107592 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 362/730] 10.222.0.58 () {56 vars in 1103 bytes} [Tue May 17 19:11:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:04.109070 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e792e97f-f285-43f9-93cb-f0dfd15a400c None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:04.119432 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a304642-e1c0-4b31-aca4-7c9dd739860e None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:04.155310 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663c8293-5546-4419-87a9-29337bf4d4bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:04.155932 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663c8293-5546-4419-87a9-29337bf4d4bc None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:04.156204 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663c8293-5546-4419-87a9-29337bf4d4bc None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:04.156473 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663c8293-5546-4419-87a9-29337bf4d4bc None admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:04.157340 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663c8293-5546-4419-87a9-29337bf4d4bc None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:04.157643 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663c8293-5546-4419-87a9-29337bf4d4bc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:04.159122 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663c8293-5546-4419-87a9-29337bf4d4bc None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:04.179314 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 369/731] 10.222.0.58 () {56 vars in 1135 bytes} [Tue May 17 19:11:04 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:04.184816 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663c8293-5546-4419-87a9-29337bf4d4bc None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:04.218742 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e792e97f-f285-43f9-93cb-f0dfd15a400c None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:04.252521 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674a239a-551b-4c1e-a758-249d9e0eb3a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:04.254360 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674a239a-551b-4c1e-a758-249d9e0eb3a7 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:04.254837 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674a239a-551b-4c1e-a758-249d9e0eb3a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:04.255314 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674a239a-551b-4c1e-a758-249d9e0eb3a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:04.266236 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674a239a-551b-4c1e-a758-249d9e0eb3a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:04.268036 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674a239a-551b-4c1e-a758-249d9e0eb3a7 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:04.283327 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663c8293-5546-4419-87a9-29337bf4d4bc None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:04.286323 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-674a239a-551b-4c1e-a758-249d9e0eb3a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:11:04.286323 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:04.286323 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:11:04.286323 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:11:04.286323 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:11:04.286323 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:11:04.286323 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:11:04.286323 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:11:04.286323 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:11:04.286323 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:11:04.286323 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:11:04.286323 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:11:04.286323 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 17 19:11:04.286323 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 17 19:11:04.286323 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 17 19:11:04.287220 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 17 19:11:04.287220 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:11:04.287220 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:11:04.287220 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:11:04.287220 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:11:04.287220 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:11:04.287220 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:11:04.287220 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:11:04.287220 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:11:04.287220 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:11:04.287220 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:11:04.287220 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:11:04.287220 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:11:04.287220 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:11:04.287220 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:11:04.287220 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:11:04.287220 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:11:04.288139 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 17 19:11:04.288139 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 17 19:11:04.288139 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:11:04.288139 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:11:04.290345 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 363/732] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:11:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:11:04.296184 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1217346-9067-4205-990c-e800b5833f89 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:04.296470 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1217346-9067-4205-990c-e800b5833f89 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:04.296745 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1217346-9067-4205-990c-e800b5833f89 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:04.298656 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 364/733] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:11:04 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:11:04.299448 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1217346-9067-4205-990c-e800b5833f89 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:04.326343 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471f5139-3387-47bc-a379-4a0e9cee5e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:04.326743 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471f5139-3387-47bc-a379-4a0e9cee5e48 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:04.327018 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471f5139-3387-47bc-a379-4a0e9cee5e48 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:04.327286 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471f5139-3387-47bc-a379-4a0e9cee5e48 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:04.393442 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471f5139-3387-47bc-a379-4a0e9cee5e48 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:04.394923 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471f5139-3387-47bc-a379-4a0e9cee5e48 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:04.403303 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1217346-9067-4205-990c-e800b5833f89 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:04.435712 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04344c7-ea64-48ea-a2ec-0a3204cf5dd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:04.436288 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04344c7-ea64-48ea-a2ec-0a3204cf5dd1 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:04.436564 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04344c7-ea64-48ea-a2ec-0a3204cf5dd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:04.436832 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04344c7-ea64-48ea-a2ec-0a3204cf5dd1 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:04.437535 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04344c7-ea64-48ea-a2ec-0a3204cf5dd1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:04.437848 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04344c7-ea64-48ea-a2ec-0a3204cf5dd1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:04.439182 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04344c7-ea64-48ea-a2ec-0a3204cf5dd1 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:04.455624 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 365/734] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 17 19:11:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:04.455624 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcba7629-8fd9-4ab1-8149-49e3511ee5a6 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:04.455624 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcba7629-8fd9-4ab1-8149-49e3511ee5a6 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:04.455624 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcba7629-8fd9-4ab1-8149-49e3511ee5a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:04.468726 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-bcba7629-8fd9-4ab1-8149-49e3511ee5a6 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:11:04.469885 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-bcba7629-8fd9-4ab1-8149-49e3511ee5a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:11:04.500155 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 370/735] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 17 19:11:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 17 19:11:04.500155 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bd82cd-7991-4025-86df-7802d4723158 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:04.500155 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bd82cd-7991-4025-86df-7802d4723158 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:04.500155 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bd82cd-7991-4025-86df-7802d4723158 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:04.500155 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 371/736] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:11:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:11:04.500155 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bd82cd-7991-4025-86df-7802d4723158 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:04.545625 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 366/737] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:11:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 17 19:11:04.546462 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcba7629-8fd9-4ab1-8149-49e3511ee5a6 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:04.613606 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bd82cd-7991-4025-86df-7802d4723158 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:04.646721 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcba7629-8fd9-4ab1-8149-49e3511ee5a6 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:04.647227 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433eb19c-7cfb-45b8-9924-07f9aee63bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:04.648103 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433eb19c-7cfb-45b8-9924-07f9aee63bca None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:04.648376 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433eb19c-7cfb-45b8-9924-07f9aee63bca None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:04.648643 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433eb19c-7cfb-45b8-9924-07f9aee63bca None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:04.655740 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433eb19c-7cfb-45b8-9924-07f9aee63bca None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:04.657175 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433eb19c-7cfb-45b8-9924-07f9aee63bca None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:04.672168 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-433eb19c-7cfb-45b8-9924-07f9aee63bca None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 17 19:11:04.672168 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:04.672168 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:11:04.672168 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:11:04.672168 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:11:04.672168 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:11:04.672168 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:11:04.672168 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:11:04.672168 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:11:04.672168 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:11:04.672168 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:11:04.672168 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:11:04.672168 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 17 19:11:04.672168 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_user(user_id) May 17 19:11:04.672168 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 17 19:11:04.672168 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 17 19:11:04.673376 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:11:04.673376 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:11:04.673376 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 17 19:11:04.673376 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 17 19:11:04.673376 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 17 19:11:04.673376 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 17 19:11:04.673376 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:11:04.673376 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:11:04.673376 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:11:04.673376 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:11:04.673376 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:11:04.673376 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:11:04.673376 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:11:04.673376 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:11:04.673376 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:11:04.673376 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:11:04.673376 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:11:04.673376 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:11:04.674618 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:11:04.674618 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:11:04.674618 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 17 19:11:04.674618 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 17 19:11:04.674618 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 17 19:11:04.674618 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 17 19:11:04.674618 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 17 19:11:04.674618 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 17 19:11:04.674618 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 17 19:11:04.674618 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:11:04.674618 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 372/738] 10.222.0.58 () {56 vars in 1117 bytes} [Tue May 17 19:11:04 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:11:04.675205 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ad9e33-1e37-44bb-a054-90d08fa944d7 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:04.675205 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ad9e33-1e37-44bb-a054-90d08fa944d7 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:04.675926 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ad9e33-1e37-44bb-a054-90d08fa944d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:04.691674 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a12765-0f86-461f-aeb6-f507c008186e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:04.691674 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a12765-0f86-461f-aeb6-f507c008186e None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:04.691674 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a12765-0f86-461f-aeb6-f507c008186e None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:04.691674 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a12765-0f86-461f-aeb6-f507c008186e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:04.692966 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-83ad9e33-1e37-44bb-a054-90d08fa944d7 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:11:04.694181 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-83ad9e33-1e37-44bb-a054-90d08fa944d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:11:04.700943 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a12765-0f86-461f-aeb6-f507c008186e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:04.702394 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a12765-0f86-461f-aeb6-f507c008186e None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:04.719665 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-b0a12765-0f86-461f-aeb6-f507c008186e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:11:04.719665 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:04.719665 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:11:04.719665 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:11:04.719665 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:11:04.719665 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:11:04.719665 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:11:04.719665 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:11:04.719665 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:11:04.719665 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:11:04.719665 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:11:04.719665 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:11:04.719665 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 17 19:11:04.719665 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 17 19:11:04.719665 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 17 19:11:04.720559 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 17 19:11:04.720559 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:11:04.720559 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:11:04.720559 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:11:04.720559 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:11:04.720559 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:11:04.720559 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:11:04.720559 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:11:04.720559 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:11:04.720559 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:11:04.720559 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:11:04.720559 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:11:04.720559 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:11:04.720559 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:11:04.720559 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:11:04.720559 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:11:04.720559 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:11:04.721727 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 17 19:11:04.721727 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 17 19:11:04.721727 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:11:04.721727 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:11:04.721727 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 367/739] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:11:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:11:04.722036 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a12765-0f86-461f-aeb6-f507c008186e None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:04.771530 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 373/740] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:11:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 17 19:11:04.771530 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ad9e33-1e37-44bb-a054-90d08fa944d7 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:04.841723 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a12765-0f86-461f-aeb6-f507c008186e None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:04.869072 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ad9e33-1e37-44bb-a054-90d08fa944d7 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:04.879945 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c036fb-385c-43ca-990a-96efbfb8308d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:04.880598 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c036fb-385c-43ca-990a-96efbfb8308d None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:04.880871 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c036fb-385c-43ca-990a-96efbfb8308d None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:04.881142 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c036fb-385c-43ca-990a-96efbfb8308d None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:04.881842 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c036fb-385c-43ca-990a-96efbfb8308d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:04.883195 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c036fb-385c-43ca-990a-96efbfb8308d None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:04.903735 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cdd8ca-cfc0-48e0-ad9f-7a53e6109b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:04.904394 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cdd8ca-cfc0-48e0-ad9f-7a53e6109b51 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:04.904669 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cdd8ca-cfc0-48e0-ad9f-7a53e6109b51 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:04.904937 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cdd8ca-cfc0-48e0-ad9f-7a53e6109b51 None admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:04.905755 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cdd8ca-cfc0-48e0-ad9f-7a53e6109b51 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:04.906054 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cdd8ca-cfc0-48e0-ad9f-7a53e6109b51 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:04.907448 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cdd8ca-cfc0-48e0-ad9f-7a53e6109b51 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:04.924097 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 374/741] 10.222.0.58 () {56 vars in 1149 bytes} [Tue May 17 19:11:04 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:04.924739 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cdd8ca-cfc0-48e0-ad9f-7a53e6109b51 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:04.967927 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 368/742] 10.222.0.58 () {60 vars in 1141 bytes} [Tue May 17 19:11:04 2022] POST /identity/v3/services => generated 251 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:11:04.968759 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c036fb-385c-43ca-990a-96efbfb8308d None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:05.025802 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cdd8ca-cfc0-48e0-ad9f-7a53e6109b51 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:05.068699 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f69de0-5c44-4df6-8c9b-5a99eb6e793c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:05.069375 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f69de0-5c44-4df6-8c9b-5a99eb6e793c None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:05.069653 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f69de0-5c44-4df6-8c9b-5a99eb6e793c None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:05.069924 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f69de0-5c44-4df6-8c9b-5a99eb6e793c None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:05.070653 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f69de0-5c44-4df6-8c9b-5a99eb6e793c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:05.070958 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f69de0-5c44-4df6-8c9b-5a99eb6e793c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:05.072389 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f69de0-5c44-4df6-8c9b-5a99eb6e793c None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:05.074181 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c036fb-385c-43ca-990a-96efbfb8308d None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:05.084857 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 375/743] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 17 19:11:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:05.085739 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f69de0-5c44-4df6-8c9b-5a99eb6e793c None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:05.114072 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af550eb0-eade-41bf-889b-f014c8b19dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:05.115215 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af550eb0-eade-41bf-889b-f014c8b19dc1 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:05.115518 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af550eb0-eade-41bf-889b-f014c8b19dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:05.115798 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af550eb0-eade-41bf-889b-f014c8b19dc1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:05.126751 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af550eb0-eade-41bf-889b-f014c8b19dc1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 17 19:11:05.139313 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-af550eb0-eade-41bf-889b-f014c8b19dc1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 17 19:11:05.139313 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:05.139313 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:11:05.139313 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:11:05.139313 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:11:05.139313 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:11:05.139313 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:11:05.139313 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:11:05.139313 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:11:05.139313 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:11:05.139313 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:11:05.139313 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:11:05.139313 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 17 19:11:05.139313 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) May 17 19:11:05.139313 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 17 19:11:05.139313 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 17 19:11:05.140367 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 17 19:11:05.140367 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 17 19:11:05.140367 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 17 19:11:05.140367 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 17 19:11:05.140367 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:11:05.140367 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:11:05.140367 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:11:05.140367 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:11:05.140367 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:11:05.140367 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:11:05.140367 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:11:05.140367 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:11:05.140367 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:11:05.140367 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:11:05.140367 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:11:05.140367 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:11:05.141219 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:11:05.141219 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:11:05.141219 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:11:05.141219 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:11:05.141219 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 17 19:11:05.141219 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 17 19:11:05.141219 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 17 19:11:05.141219 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 17 19:11:05.141219 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 17 19:11:05.141219 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 17 19:11:05.141219 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 17 19:11:05.141219 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:11:05.141219 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 369/744] 10.222.0.58 () {56 vars in 1096 bytes} [Tue May 17 19:11:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:11:05.142579 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f705f783-0ea0-4614-a188-45f5719157ea None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:05.142579 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f705f783-0ea0-4614-a188-45f5719157ea None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:05.142579 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f705f783-0ea0-4614-a188-45f5719157ea None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:05.143804 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 370/745] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:11:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:11:05.145032 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f705f783-0ea0-4614-a188-45f5719157ea None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:05.203938 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f69de0-5c44-4df6-8c9b-5a99eb6e793c None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:05.237794 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35d2f52-c96e-43b0-88a9-4571bb4e7121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:05.238467 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35d2f52-c96e-43b0-88a9-4571bb4e7121 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:05.238751 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35d2f52-c96e-43b0-88a9-4571bb4e7121 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:05.239022 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35d2f52-c96e-43b0-88a9-4571bb4e7121 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:05.254370 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f705f783-0ea0-4614-a188-45f5719157ea None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:05.290022 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a349c0b-6910-49a9-aeb4-e6f07cf0e71c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:05.290859 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a349c0b-6910-49a9-aeb4-e6f07cf0e71c None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:05.291138 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a349c0b-6910-49a9-aeb4-e6f07cf0e71c None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:05.291400 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a349c0b-6910-49a9-aeb4-e6f07cf0e71c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:05.297946 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a349c0b-6910-49a9-aeb4-e6f07cf0e71c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:05.299368 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a349c0b-6910-49a9-aeb4-e6f07cf0e71c None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:05.304096 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35d2f52-c96e-43b0-88a9-4571bb4e7121 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:05.305571 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35d2f52-c96e-43b0-88a9-4571bb4e7121 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:05.313143 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-0a349c0b-6910-49a9-aeb4-e6f07cf0e71c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 17 19:11:05.313143 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:05.313143 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:11:05.313143 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:11:05.313143 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:11:05.313143 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:11:05.313143 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:11:05.313143 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:11:05.313143 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:11:05.313143 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:11:05.313143 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:11:05.313143 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:11:05.313143 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 17 19:11:05.313143 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(project_id) May 17 19:11:05.313143 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 17 19:11:05.314064 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 17 19:11:05.314064 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:11:05.314064 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:11:05.314064 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:11:05.314064 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:11:05.314064 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:11:05.314064 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:11:05.314064 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:11:05.314064 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:11:05.314064 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:11:05.314064 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:11:05.314064 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:11:05.314064 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:11:05.314064 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:11:05.314064 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:11:05.314064 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:11:05.314064 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:11:05.314960 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 17 19:11:05.314960 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 17 19:11:05.314960 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 17 19:11:05.314960 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 17 19:11:05.314960 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 17 19:11:05.314960 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 17 19:11:05.314960 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 17 19:11:05.314960 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:11:05.314960 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 371/746] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 17 19:11:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:11:05.315470 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a349c0b-6910-49a9-aeb4-e6f07cf0e71c None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:05.410628 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 376/747] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 17 19:11:05 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 17 19:11:05.410628 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5933b9f5-1dfe-47a8-97e7-b9ae153639c4 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:05.410628 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5933b9f5-1dfe-47a8-97e7-b9ae153639c4 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:05.410628 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5933b9f5-1dfe-47a8-97e7-b9ae153639c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:05.419114 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a349c0b-6910-49a9-aeb4-e6f07cf0e71c None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:05.424336 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-5933b9f5-1dfe-47a8-97e7-b9ae153639c4 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:11:05.426253 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-5933b9f5-1dfe-47a8-97e7-b9ae153639c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:11:05.459622 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cb2ed2-e9dc-4d39-9b83-141bc0874fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:05.460972 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cb2ed2-e9dc-4d39-9b83-141bc0874fde None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:05.461249 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cb2ed2-e9dc-4d39-9b83-141bc0874fde None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:05.461516 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cb2ed2-e9dc-4d39-9b83-141bc0874fde None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:05.462251 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cb2ed2-e9dc-4d39-9b83-141bc0874fde None admin] RBAC: Adding query filter params (name=admin) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:05.462549 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cb2ed2-e9dc-4d39-9b83-141bc0874fde None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:05.463944 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cb2ed2-e9dc-4d39-9b83-141bc0874fde None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:05.474521 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 372/748] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 17 19:11:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:05.475651 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cb2ed2-e9dc-4d39-9b83-141bc0874fde None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:05.499953 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 377/749] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:11:05 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 17 19:11:05.499953 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5933b9f5-1dfe-47a8-97e7-b9ae153639c4 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:05.579934 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cb2ed2-e9dc-4d39-9b83-141bc0874fde None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:05.601197 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5933b9f5-1dfe-47a8-97e7-b9ae153639c4 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:05.612604 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5b16bc-f130-451e-b86c-53a2d8221f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:05.613280 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5b16bc-f130-451e-b86c-53a2d8221f97 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:05.613560 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5b16bc-f130-451e-b86c-53a2d8221f97 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:05.613834 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5b16bc-f130-451e-b86c-53a2d8221f97 None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:05.614624 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5b16bc-f130-451e-b86c-53a2d8221f97 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:05.614914 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5b16bc-f130-451e-b86c-53a2d8221f97 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:05.616382 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5b16bc-f130-451e-b86c-53a2d8221f97 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:05.627892 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 373/750] 10.222.0.58 () {56 vars in 1148 bytes} [Tue May 17 19:11:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:05.629232 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5b16bc-f130-451e-b86c-53a2d8221f97 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:05.643754 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da34cff5-e28e-4a82-a88d-0d8b3f2baab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:05.648638 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da34cff5-e28e-4a82-a88d-0d8b3f2baab6 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:05.648638 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da34cff5-e28e-4a82-a88d-0d8b3f2baab6 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:05.648638 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da34cff5-e28e-4a82-a88d-0d8b3f2baab6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:05.655782 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da34cff5-e28e-4a82-a88d-0d8b3f2baab6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:05.657221 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da34cff5-e28e-4a82-a88d-0d8b3f2baab6 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:05.674437 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-da34cff5-e28e-4a82-a88d-0d8b3f2baab6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:11:05.674437 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:05.674437 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:11:05.674437 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:11:05.674437 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:11:05.674437 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:11:05.674437 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:11:05.674437 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:11:05.674437 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:11:05.674437 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:11:05.674437 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:11:05.674437 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:11:05.674437 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 17 19:11:05.674437 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 17 19:11:05.674437 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 17 19:11:05.681324 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 17 19:11:05.681324 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:11:05.681324 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:11:05.681324 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:11:05.681324 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:11:05.681324 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:11:05.681324 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:11:05.681324 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:11:05.681324 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:11:05.681324 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:11:05.681324 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:11:05.681324 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:11:05.681324 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:11:05.681324 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:11:05.681324 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:11:05.681324 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:11:05.681324 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:11:05.682457 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 17 19:11:05.682457 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 17 19:11:05.682457 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:11:05.682457 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:11:05.682457 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 378/751] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:11:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:11:05.682457 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da34cff5-e28e-4a82-a88d-0d8b3f2baab6 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:05.726356 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5b16bc-f130-451e-b86c-53a2d8221f97 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:05.758582 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9995c2-7cb0-4e4a-a407-d32c7c44e3bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:05.759368 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9995c2-7cb0-4e4a-a407-d32c7c44e3bf None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:05.759676 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9995c2-7cb0-4e4a-a407-d32c7c44e3bf None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:05.759947 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9995c2-7cb0-4e4a-a407-d32c7c44e3bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:05.769768 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9995c2-7cb0-4e4a-a407-d32c7c44e3bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:05.771210 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9995c2-7cb0-4e4a-a407-d32c7c44e3bf None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:05.788835 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-6e9995c2-7cb0-4e4a-a407-d32c7c44e3bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:11:05.788835 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:05.788835 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:11:05.788835 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:11:05.788835 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:11:05.788835 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:11:05.788835 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:11:05.788835 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:11:05.788835 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:11:05.788835 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:11:05.788835 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:11:05.788835 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:11:05.788835 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 17 19:11:05.788835 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 17 19:11:05.788835 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 17 19:11:05.789759 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 17 19:11:05.789759 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:11:05.789759 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:11:05.789759 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:11:05.789759 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:11:05.789759 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:11:05.789759 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:11:05.789759 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:11:05.789759 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:11:05.789759 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:11:05.789759 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:11:05.789759 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:11:05.789759 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:11:05.789759 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:11:05.789759 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:11:05.789759 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:11:05.789759 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:11:05.790700 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 17 19:11:05.790700 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 17 19:11:05.790700 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:11:05.790700 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:11:05.791172 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da34cff5-e28e-4a82-a88d-0d8b3f2baab6 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:05.794554 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 374/752] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:11:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:11:05.795146 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9995c2-7cb0-4e4a-a407-d32c7c44e3bf None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:05.823582 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3693050f-7482-42fe-a7a9-7efa0bfbe6c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:05.824281 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3693050f-7482-42fe-a7a9-7efa0bfbe6c7 None admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:05.824512 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3693050f-7482-42fe-a7a9-7efa0bfbe6c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:05.824780 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3693050f-7482-42fe-a7a9-7efa0bfbe6c7 None admin] PATH_INFO: `/v3/services` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:05.825633 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3693050f-7482-42fe-a7a9-7efa0bfbe6c7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:05.826997 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3693050f-7482-42fe-a7a9-7efa0bfbe6c7 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:05.890760 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9995c2-7cb0-4e4a-a407-d32c7c44e3bf None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:05.899688 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 379/753] 10.222.0.58 () {60 vars in 1141 bytes} [Tue May 17 19:11:05 2022] POST /identity/v3/services => generated 240 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:11:05.900665 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3693050f-7482-42fe-a7a9-7efa0bfbe6c7 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:05.946833 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b8873a-33f6-4258-9302-1a17baace6c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:05.946833 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b8873a-33f6-4258-9302-1a17baace6c5 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:05.946833 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b8873a-33f6-4258-9302-1a17baace6c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:05.946833 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b8873a-33f6-4258-9302-1a17baace6c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:05.949351 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b8873a-33f6-4258-9302-1a17baace6c5 None admin] RBAC: Adding query filter params (role.id=60af5088793e4379b88821a86552a344, scope.project.id=1a9d3a0631284c9fb113266036993e2d, user.id=b0cb292002434d30bc81f370d120a2a2) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:05.949709 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b8873a-33f6-4258-9302-1a17baace6c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:05.951370 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b8873a-33f6-4258-9302-1a17baace6c5 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:05.964113 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 375/754] 10.222.0.58 () {56 vars in 1374 bytes} [Tue May 17 19:11:05 2022] GET /identity/v3/role_assignments?user.id=b0cb292002434d30bc81f370d120a2a2&scope.project.id=1a9d3a0631284c9fb113266036993e2d&role.id=60af5088793e4379b88821a86552a344 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:05.965085 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b8873a-33f6-4258-9302-1a17baace6c5 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:06.017352 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3693050f-7482-42fe-a7a9-7efa0bfbe6c7 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:06.056413 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fd6c67-6a4c-4e28-86fb-51c2fa3d577a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:06.057378 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fd6c67-6a4c-4e28-86fb-51c2fa3d577a None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:06.057912 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fd6c67-6a4c-4e28-86fb-51c2fa3d577a None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:06.058330 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fd6c67-6a4c-4e28-86fb-51c2fa3d577a None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:06.059329 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fd6c67-6a4c-4e28-86fb-51c2fa3d577a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:06.059831 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fd6c67-6a4c-4e28-86fb-51c2fa3d577a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:06.061382 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fd6c67-6a4c-4e28-86fb-51c2fa3d577a None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:06.075241 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 380/755] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 17 19:11:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:06.080873 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fd6c67-6a4c-4e28-86fb-51c2fa3d577a None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:06.145269 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b8873a-33f6-4258-9302-1a17baace6c5 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:06.194183 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b469e0d-6574-4339-8e7c-82509a2fa005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:06.195250 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b469e0d-6574-4339-8e7c-82509a2fa005 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:06.195782 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b469e0d-6574-4339-8e7c-82509a2fa005 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:06.196257 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b469e0d-6574-4339-8e7c-82509a2fa005 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:06.197190 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b469e0d-6574-4339-8e7c-82509a2fa005 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:06.197715 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b469e0d-6574-4339-8e7c-82509a2fa005 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:06.199328 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b469e0d-6574-4339-8e7c-82509a2fa005 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:06.210761 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fd6c67-6a4c-4e28-86fb-51c2fa3d577a None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:06.213050 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 376/756] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 17 19:11:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:06.219850 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b469e0d-6574-4339-8e7c-82509a2fa005 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:06.258140 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a011aa-da9b-466c-a7f0-6f2d82a4a169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:06.259622 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a011aa-da9b-466c-a7f0-6f2d82a4a169 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:06.259622 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a011aa-da9b-466c-a7f0-6f2d82a4a169 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:06.259622 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a011aa-da9b-466c-a7f0-6f2d82a4a169 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:06.269194 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a011aa-da9b-466c-a7f0-6f2d82a4a169 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:06.270944 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a011aa-da9b-466c-a7f0-6f2d82a4a169 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:06.285993 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-65a011aa-da9b-466c-a7f0-6f2d82a4a169 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 17 19:11:06.285993 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:06.285993 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:11:06.285993 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:11:06.285993 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:11:06.285993 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:11:06.285993 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:11:06.285993 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:11:06.285993 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:11:06.285993 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:11:06.285993 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:11:06.285993 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:11:06.285993 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 17 19:11:06.285993 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(project_id) May 17 19:11:06.285993 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 17 19:11:06.286952 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 17 19:11:06.286952 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:11:06.286952 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:11:06.286952 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:11:06.286952 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:11:06.286952 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:11:06.286952 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:11:06.286952 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:11:06.286952 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:11:06.286952 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:11:06.286952 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:11:06.286952 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:11:06.286952 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:11:06.286952 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:11:06.286952 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:11:06.286952 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:11:06.286952 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:11:06.287982 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 17 19:11:06.287982 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 17 19:11:06.287982 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 17 19:11:06.287982 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 17 19:11:06.287982 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 17 19:11:06.287982 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 17 19:11:06.287982 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 17 19:11:06.287982 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:11:06.290424 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 381/757] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 17 19:11:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 17 19:11:06.296052 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a011aa-da9b-466c-a7f0-6f2d82a4a169 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:06.339965 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b469e0d-6574-4339-8e7c-82509a2fa005 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:06.405750 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23053644-50b0-46a7-92c5-9ce9f026f33c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:06.408048 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23053644-50b0-46a7-92c5-9ce9f026f33c None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:06.408539 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23053644-50b0-46a7-92c5-9ce9f026f33c None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:06.409031 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23053644-50b0-46a7-92c5-9ce9f026f33c None admin] PATH_INFO: `/v3/users/nova` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:06.421564 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23053644-50b0-46a7-92c5-9ce9f026f33c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:06.428288 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23053644-50b0-46a7-92c5-9ce9f026f33c None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:06.454913 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-23053644-50b0-46a7-92c5-9ce9f026f33c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 17 19:11:06.454913 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:06.454913 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:11:06.454913 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:11:06.454913 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:11:06.454913 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:11:06.454913 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:11:06.454913 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:11:06.454913 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:11:06.454913 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:11:06.454913 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:11:06.454913 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:11:06.454913 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 17 19:11:06.454913 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_user(user_id) May 17 19:11:06.454913 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 17 19:11:06.454913 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 17 19:11:06.455961 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:11:06.455961 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:11:06.455961 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 17 19:11:06.455961 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 17 19:11:06.455961 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 17 19:11:06.455961 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 17 19:11:06.455961 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:11:06.455961 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:11:06.455961 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:11:06.455961 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:11:06.455961 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:11:06.455961 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:11:06.455961 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:11:06.455961 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:11:06.455961 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:11:06.455961 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:11:06.455961 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:11:06.455961 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:11:06.457030 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:11:06.457030 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:11:06.457030 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 17 19:11:06.457030 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 17 19:11:06.457030 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 17 19:11:06.457030 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 17 19:11:06.457030 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 17 19:11:06.457030 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 17 19:11:06.457030 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 17 19:11:06.457030 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:11:06.457030 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 377/758] 10.222.0.58 () {56 vars in 1093 bytes} [Tue May 17 19:11:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:11:06.463613 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23053644-50b0-46a7-92c5-9ce9f026f33c None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:06.481682 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a011aa-da9b-466c-a7f0-6f2d82a4a169 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:06.552709 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e185e8-7017-49a6-aa97-edce7e3e833c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:06.553301 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e185e8-7017-49a6-aa97-edce7e3e833c None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:06.553641 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e185e8-7017-49a6-aa97-edce7e3e833c None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:06.553917 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e185e8-7017-49a6-aa97-edce7e3e833c None admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:06.555466 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e185e8-7017-49a6-aa97-edce7e3e833c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:06.555855 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e185e8-7017-49a6-aa97-edce7e3e833c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:06.557305 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e185e8-7017-49a6-aa97-edce7e3e833c None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:06.580774 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 382/759] 10.222.0.58 () {56 vars in 1148 bytes} [Tue May 17 19:11:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:11:06.587349 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e185e8-7017-49a6-aa97-edce7e3e833c None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:06.619241 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23053644-50b0-46a7-92c5-9ce9f026f33c None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:06.654668 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a577e07-bf48-41f1-8958-97a56db699d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:06.655338 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a577e07-bf48-41f1-8958-97a56db699d6 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:06.655796 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a577e07-bf48-41f1-8958-97a56db699d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:06.656069 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a577e07-bf48-41f1-8958-97a56db699d6 None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:06.657411 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a577e07-bf48-41f1-8958-97a56db699d6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:06.657719 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a577e07-bf48-41f1-8958-97a56db699d6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:06.660218 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a577e07-bf48-41f1-8958-97a56db699d6 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:06.682930 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 378/760] 10.222.0.58 () {56 vars in 1133 bytes} [Tue May 17 19:11:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:11:06.689077 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a577e07-bf48-41f1-8958-97a56db699d6 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:06.696796 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e185e8-7017-49a6-aa97-edce7e3e833c None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:06.769790 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5180b35a-cf41-4ed8-a2db-ee704a2496c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:06.770834 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5180b35a-cf41-4ed8-a2db-ee704a2496c2 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:06.771309 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5180b35a-cf41-4ed8-a2db-ee704a2496c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:06.771801 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5180b35a-cf41-4ed8-a2db-ee704a2496c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:06.772815 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5180b35a-cf41-4ed8-a2db-ee704a2496c2 None admin] RBAC: Adding query filter params (role.id=1754a95b778c4e20b53b20cb52ada92b, scope.project.id=1a9d3a0631284c9fb113266036993e2d, user.id=2b90b57c542440448579b3aadda8b241) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:06.773296 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5180b35a-cf41-4ed8-a2db-ee704a2496c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:06.775039 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5180b35a-cf41-4ed8-a2db-ee704a2496c2 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:06.788349 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 383/761] 10.222.0.58 () {56 vars in 1374 bytes} [Tue May 17 19:11:06 2022] GET /identity/v3/role_assignments?user.id=2b90b57c542440448579b3aadda8b241&scope.project.id=1a9d3a0631284c9fb113266036993e2d&role.id=1754a95b778c4e20b53b20cb52ada92b => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:06.832148 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a577e07-bf48-41f1-8958-97a56db699d6 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:06.875481 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efc1886-5624-4160-a7f4-5fa81ee4bb94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:06.876331 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efc1886-5624-4160-a7f4-5fa81ee4bb94 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:06.876706 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efc1886-5624-4160-a7f4-5fa81ee4bb94 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:06.876954 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efc1886-5624-4160-a7f4-5fa81ee4bb94 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:06.892984 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efc1886-5624-4160-a7f4-5fa81ee4bb94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:06.895094 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efc1886-5624-4160-a7f4-5fa81ee4bb94 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:06.923414 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-2efc1886-5624-4160-a7f4-5fa81ee4bb94 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:11:06.923414 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:06.923414 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:11:06.923414 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:11:06.923414 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:11:06.923414 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:11:06.923414 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:11:06.923414 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:11:06.923414 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:11:06.923414 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:11:06.923414 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:11:06.923414 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:11:06.923414 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 17 19:11:06.923414 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 17 19:11:06.923414 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 17 19:11:06.924455 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 17 19:11:06.924455 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:11:06.924455 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:11:06.924455 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:11:06.924455 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:11:06.924455 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:11:06.924455 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:11:06.924455 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:11:06.924455 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:11:06.924455 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:11:06.924455 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:11:06.924455 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:11:06.924455 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:11:06.924455 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:11:06.924455 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:11:06.924455 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:11:06.924455 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:11:06.925925 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 17 19:11:06.925925 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 17 19:11:06.925925 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:11:06.925925 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:11:06.925925 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 379/762] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:11:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:11:06.932198 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5180b35a-cf41-4ed8-a2db-ee704a2496c2 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:07.079078 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5180b35a-cf41-4ed8-a2db-ee704a2496c2 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:07.122599 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0d139f-1b67-4816-bf54-362db8714695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:07.123197 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0d139f-1b67-4816-bf54-362db8714695 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:07.123565 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0d139f-1b67-4816-bf54-362db8714695 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:07.123844 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0d139f-1b67-4816-bf54-362db8714695 None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:07.124641 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0d139f-1b67-4816-bf54-362db8714695 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:07.124950 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0d139f-1b67-4816-bf54-362db8714695 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:07.126465 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0d139f-1b67-4816-bf54-362db8714695 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:07.148604 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 384/763] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 17 19:11:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:07.154219 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efc1886-5624-4160-a7f4-5fa81ee4bb94 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:07.299025 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efc1886-5624-4160-a7f4-5fa81ee4bb94 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:07.324877 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a53708-7aeb-4006-a68e-53d9e688f490 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:07.325498 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a53708-7aeb-4006-a68e-53d9e688f490 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:07.325972 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a53708-7aeb-4006-a68e-53d9e688f490 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:07.327941 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 385/764] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:11:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:11:07.333260 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ff50c5-1e32-4aed-bee7-b0e99b26bd9e None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:07.333565 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ff50c5-1e32-4aed-bee7-b0e99b26bd9e None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:07.333808 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ff50c5-1e32-4aed-bee7-b0e99b26bd9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:07.342954 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5bef15-42cf-44d9-b86c-e86346478cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:07.343737 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5bef15-42cf-44d9-b86c-e86346478cc4 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:07.344013 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5bef15-42cf-44d9-b86c-e86346478cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:07.344283 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5bef15-42cf-44d9-b86c-e86346478cc4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:07.351515 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5bef15-42cf-44d9-b86c-e86346478cc4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:07.352986 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5bef15-42cf-44d9-b86c-e86346478cc4 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:07.367169 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-1b5bef15-42cf-44d9-b86c-e86346478cc4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 17 19:11:07.367169 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:07.367169 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:11:07.367169 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:11:07.367169 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:11:07.367169 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:11:07.367169 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:11:07.367169 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:11:07.367169 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:11:07.367169 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:11:07.367169 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:11:07.367169 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:11:07.367169 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 17 19:11:07.367169 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(project_id) May 17 19:11:07.367169 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 17 19:11:07.368149 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 17 19:11:07.368149 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:11:07.368149 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:11:07.368149 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:11:07.368149 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:11:07.368149 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:11:07.368149 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:11:07.368149 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:11:07.368149 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:11:07.368149 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:11:07.368149 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:11:07.368149 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:11:07.368149 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:11:07.368149 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:11:07.368149 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:11:07.368149 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:11:07.368149 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:11:07.369073 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 17 19:11:07.369073 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 17 19:11:07.369073 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 17 19:11:07.369073 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 17 19:11:07.369073 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 17 19:11:07.369073 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 17 19:11:07.369073 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 17 19:11:07.369073 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:11:07.369073 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 380/765] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 17 19:11:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:11:07.372509 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-c6ff50c5-1e32-4aed-bee7-b0e99b26bd9e None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:11:07.373736 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-c6ff50c5-1e32-4aed-bee7-b0e99b26bd9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:11:07.376348 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5bef15-42cf-44d9-b86c-e86346478cc4 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:07.512149 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 386/766] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:11:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 17 19:11:07.538198 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5bef15-42cf-44d9-b86c-e86346478cc4 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:07.557529 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ff50c5-1e32-4aed-bee7-b0e99b26bd9e None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:07.601088 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f3aa29-740e-43c4-b1b0-9c0b845d42fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:07.601802 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f3aa29-740e-43c4-b1b0-9c0b845d42fb None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:07.602077 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f3aa29-740e-43c4-b1b0-9c0b845d42fb None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:07.602343 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f3aa29-740e-43c4-b1b0-9c0b845d42fb None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:07.604207 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f3aa29-740e-43c4-b1b0-9c0b845d42fb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:07.604510 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f3aa29-740e-43c4-b1b0-9c0b845d42fb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:07.605940 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f3aa29-740e-43c4-b1b0-9c0b845d42fb None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:07.633843 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 381/767] 10.222.0.58 () {56 vars in 1148 bytes} [Tue May 17 19:11:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:07.639911 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f3aa29-740e-43c4-b1b0-9c0b845d42fb None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:07.765654 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ff50c5-1e32-4aed-bee7-b0e99b26bd9e None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:07.781927 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f3aa29-740e-43c4-b1b0-9c0b845d42fb None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:07.824935 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88710ab0-0de0-429d-b5f4-0eaf782fd021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:07.825807 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88710ab0-0de0-429d-b5f4-0eaf782fd021 None admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:07.826138 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88710ab0-0de0-429d-b5f4-0eaf782fd021 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:07.826415 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88710ab0-0de0-429d-b5f4-0eaf782fd021 None admin] PATH_INFO: `/v3/projects/1a9d3a0631284c9fb113266036993e2d/users/65c1f878b0dc4d63a4f5dcb12fac1dc1/roles/60af5088793e4379b88821a86552a344` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:07.829089 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88710ab0-0de0-429d-b5f4-0eaf782fd021 None admin] RBAC: Authorizing `identity:create_grant(project_id=1a9d3a0631284c9fb113266036993e2d, user_id=65c1f878b0dc4d63a4f5dcb12fac1dc1, role_id=60af5088793e4379b88821a86552a344)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:07.830510 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88710ab0-0de0-429d-b5f4-0eaf782fd021 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:07.836139 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac907bad-623e-4f60-843b-af102fcedba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:07.836943 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac907bad-623e-4f60-843b-af102fcedba4 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:07.837215 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac907bad-623e-4f60-843b-af102fcedba4 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:07.837481 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac907bad-623e-4f60-843b-af102fcedba4 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:07.838186 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac907bad-623e-4f60-843b-af102fcedba4 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:07.840138 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac907bad-623e-4f60-843b-af102fcedba4 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:07.853403 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-ac907bad-623e-4f60-843b-af102fcedba4 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 17 19:11:07.853403 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:07.853403 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 17 19:11:07.853403 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 17 19:11:07.853403 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 17 19:11:07.853403 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 17 19:11:07.853403 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 17 19:11:07.853403 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 17 19:11:07.853403 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 17 19:11:07.853403 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:11:07.853403 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 17 19:11:07.853403 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:11:07.853403 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:07.853403 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:11:07.853403 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:11:07.853403 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:11:07.855564 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:11:07.855564 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:11:07.855564 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:11:07.855564 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:11:07.855564 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:11:07.855564 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:11:07.855564 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:11:07.855564 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 17 19:11:07.855564 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_service(service_id) May 17 19:11:07.855564 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 17 19:11:07.855564 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 17 19:11:07.855564 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:11:07.855564 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:11:07.855564 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:11:07.855564 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:11:07.855564 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:11:07.857654 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:11:07.857654 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:11:07.857654 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:11:07.857654 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:11:07.857654 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:11:07.857654 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:11:07.857654 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:11:07.857654 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:11:07.857654 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:11:07.857654 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:11:07.857654 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:11:07.857654 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 17 19:11:07.857654 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 17 19:11:07.857654 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 17 19:11:07.857654 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:11:07.857654 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 387/768] 10.222.0.58 () {56 vars in 1129 bytes} [Tue May 17 19:11:07 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 17 19:11:07.862116 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac907bad-623e-4f60-843b-af102fcedba4 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:07.907160 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 382/769] 10.222.0.58 () {58 vars in 1439 bytes} [Tue May 17 19:11:07 2022] PUT /identity/v3/projects/1a9d3a0631284c9fb113266036993e2d/users/65c1f878b0dc4d63a4f5dcb12fac1dc1/roles/60af5088793e4379b88821a86552a344 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:11:07.999963 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac907bad-623e-4f60-843b-af102fcedba4 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:08.057632 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce607fa0-d30d-417b-913f-33efad441fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:08.058307 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce607fa0-d30d-417b-913f-33efad441fc6 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:08.058588 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce607fa0-d30d-417b-913f-33efad441fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:08.058859 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce607fa0-d30d-417b-913f-33efad441fc6 None admin] PATH_INFO: `/v3/services` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:08.059595 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce607fa0-d30d-417b-913f-33efad441fc6 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:08.059899 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce607fa0-d30d-417b-913f-33efad441fc6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:08.061259 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce607fa0-d30d-417b-913f-33efad441fc6 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:08.069763 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 388/770] 10.222.0.58 () {56 vars in 1124 bytes} [Tue May 17 19:11:07 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:11:08.075788 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88710ab0-0de0-429d-b5f4-0eaf782fd021 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:08.225043 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88710ab0-0de0-429d-b5f4-0eaf782fd021 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:08.265501 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e00b42a-81ee-4ef7-83d4-ce62c8d4f75e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:08.266183 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e00b42a-81ee-4ef7-83d4-ce62c8d4f75e None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:08.266462 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e00b42a-81ee-4ef7-83d4-ce62c8d4f75e None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:08.266749 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e00b42a-81ee-4ef7-83d4-ce62c8d4f75e None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:08.267436 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e00b42a-81ee-4ef7-83d4-ce62c8d4f75e None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:08.267766 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e00b42a-81ee-4ef7-83d4-ce62c8d4f75e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:08.269707 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e00b42a-81ee-4ef7-83d4-ce62c8d4f75e None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:08.276557 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 383/771] 10.222.0.58 () {56 vars in 1124 bytes} [Tue May 17 19:11:08 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:08.282448 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce607fa0-d30d-417b-913f-33efad441fc6 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:08.415413 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce607fa0-d30d-417b-913f-33efad441fc6 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:08.462724 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f850f3d4-a65e-4382-b351-f05ca8c0ce70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:08.463855 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f850f3d4-a65e-4382-b351-f05ca8c0ce70 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:08.464417 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f850f3d4-a65e-4382-b351-f05ca8c0ce70 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:08.464919 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f850f3d4-a65e-4382-b351-f05ca8c0ce70 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:08.466031 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f850f3d4-a65e-4382-b351-f05ca8c0ce70 None admin] RBAC: Adding query filter params (interface=public, service_id=6507370f37c845aea6af543723e70550, region_id=RegionOne) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:08.466570 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f850f3d4-a65e-4382-b351-f05ca8c0ce70 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:08.468402 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f850f3d4-a65e-4382-b351-f05ca8c0ce70 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:08.482683 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 389/772] 10.222.0.58 () {56 vars in 1251 bytes} [Tue May 17 19:11:08 2022] GET /identity/v3/endpoints?service_id=6507370f37c845aea6af543723e70550&interface=public®ion_id=RegionOne => generated 194 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:08.488335 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e00b42a-81ee-4ef7-83d4-ce62c8d4f75e None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:08.591970 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e00b42a-81ee-4ef7-83d4-ce62c8d4f75e None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:08.627024 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777feee6-cda2-48fe-bc31-6cbdff0d2068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:08.627718 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777feee6-cda2-48fe-bc31-6cbdff0d2068 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:08.628002 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777feee6-cda2-48fe-bc31-6cbdff0d2068 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:08.628805 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777feee6-cda2-48fe-bc31-6cbdff0d2068 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:08.629485 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777feee6-cda2-48fe-bc31-6cbdff0d2068 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:08.630876 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777feee6-cda2-48fe-bc31-6cbdff0d2068 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:08.634350 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fb6461-dc65-4c56-b1d9-6ed734fa0150 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:08.634933 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fb6461-dc65-4c56-b1d9-6ed734fa0150 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:08.635407 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fb6461-dc65-4c56-b1d9-6ed734fa0150 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:08.637022 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 390/773] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:11:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 17 19:11:08.638104 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 384/774] 10.222.0.58 () {56 vars in 1087 bytes} [Tue May 17 19:11:08 2022] GET /identity/v3/services => generated 772 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:08.657504 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78b0b53-f633-4887-b6af-177b3abb91ae None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:08.657826 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78b0b53-f633-4887-b6af-177b3abb91ae None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:08.658069 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78b0b53-f633-4887-b6af-177b3abb91ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:08.683909 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-e78b0b53-f633-4887-b6af-177b3abb91ae None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:11:08.689298 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-e78b0b53-f633-4887-b6af-177b3abb91ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:11:08.780170 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 385/775] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:11:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 17 19:11:08.804471 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fb6461-dc65-4c56-b1d9-6ed734fa0150 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:08.827671 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77cd616-6de4-4148-8731-b9993361d655 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:08.827984 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77cd616-6de4-4148-8731-b9993361d655 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:08.828225 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77cd616-6de4-4148-8731-b9993361d655 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:08.829866 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 386/776] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:11:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:11:08.835318 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8148eab0-a333-4b8e-976f-72eb8a74b2cd None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:08.835636 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8148eab0-a333-4b8e-976f-72eb8a74b2cd None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:08.835918 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8148eab0-a333-4b8e-976f-72eb8a74b2cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:08.859789 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-8148eab0-a333-4b8e-976f-72eb8a74b2cd None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:11:08.860993 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-8148eab0-a333-4b8e-976f-72eb8a74b2cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:11:08.957315 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fb6461-dc65-4c56-b1d9-6ed734fa0150 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:08.957625 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 387/777] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:11:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 17 19:11:08.982709 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8148eab0-a333-4b8e-976f-72eb8a74b2cd None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:08.998407 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b566075b-6177-431a-82ba-7416c12cab96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:08.999761 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b566075b-6177-431a-82ba-7416c12cab96 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:09.000414 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b566075b-6177-431a-82ba-7416c12cab96 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:09.000868 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b566075b-6177-431a-82ba-7416c12cab96 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:09.015477 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b566075b-6177-431a-82ba-7416c12cab96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 17 19:11:09.024470 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-b566075b-6177-431a-82ba-7416c12cab96 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 17 19:11:09.024470 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:09.024470 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:11:09.024470 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:11:09.024470 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:11:09.024470 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:11:09.024470 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:11:09.024470 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:11:09.024470 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:11:09.024470 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:11:09.024470 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:11:09.024470 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:11:09.024470 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 17 19:11:09.024470 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) May 17 19:11:09.024470 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 17 19:11:09.024470 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 17 19:11:09.025548 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 17 19:11:09.025548 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 17 19:11:09.025548 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 17 19:11:09.025548 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 17 19:11:09.025548 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:11:09.025548 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:11:09.025548 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:11:09.025548 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:11:09.025548 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:11:09.025548 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:11:09.025548 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:11:09.025548 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:11:09.025548 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:11:09.025548 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:11:09.025548 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:11:09.025548 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:11:09.026399 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:11:09.026399 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:11:09.026399 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:11:09.026399 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:11:09.026399 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 17 19:11:09.026399 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 17 19:11:09.026399 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 17 19:11:09.026399 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 17 19:11:09.026399 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 17 19:11:09.026399 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 17 19:11:09.026399 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 17 19:11:09.026399 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:11:09.029134 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 391/778] 10.222.0.58 () {56 vars in 1096 bytes} [Tue May 17 19:11:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:11:09.035127 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b566075b-6177-431a-82ba-7416c12cab96 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:09.141606 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8148eab0-a333-4b8e-976f-72eb8a74b2cd None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:09.147479 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b566075b-6177-431a-82ba-7416c12cab96 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:09.184562 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deb3026-427f-401b-b3e9-049b99cc8942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:09.185580 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deb3026-427f-401b-b3e9-049b99cc8942 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:09.186313 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deb3026-427f-401b-b3e9-049b99cc8942 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:09.186764 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deb3026-427f-401b-b3e9-049b99cc8942 None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:09.187712 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0deb3026-427f-401b-b3e9-049b99cc8942 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:09.188195 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0deb3026-427f-401b-b3e9-049b99cc8942 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:09.190208 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0deb3026-427f-401b-b3e9-049b99cc8942 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:09.202066 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 392/779] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 17 19:11:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:09.211521 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deb3026-427f-401b-b3e9-049b99cc8942 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:09.211673 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba796136-e7d9-4445-8d54-c88d9c866c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:09.211673 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba796136-e7d9-4445-8d54-c88d9c866c9c None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:09.211673 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba796136-e7d9-4445-8d54-c88d9c866c9c None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:09.211673 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba796136-e7d9-4445-8d54-c88d9c866c9c None admin] PATH_INFO: `/v3/services/network` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:09.212242 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba796136-e7d9-4445-8d54-c88d9c866c9c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:09.213747 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba796136-e7d9-4445-8d54-c88d9c866c9c None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:09.231086 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-ba796136-e7d9-4445-8d54-c88d9c866c9c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 17 19:11:09.231086 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:09.231086 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 17 19:11:09.231086 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 17 19:11:09.231086 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 17 19:11:09.231086 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 17 19:11:09.231086 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 17 19:11:09.231086 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 17 19:11:09.231086 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 17 19:11:09.231086 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:11:09.231086 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 17 19:11:09.231086 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:11:09.231086 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:09.231086 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:11:09.231086 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:11:09.231086 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:11:09.231086 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:11:09.232176 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:11:09.232176 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:11:09.232176 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:11:09.232176 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:11:09.232176 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:11:09.232176 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:11:09.232176 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 17 19:11:09.232176 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(service_id) May 17 19:11:09.232176 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 17 19:11:09.232176 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 17 19:11:09.232176 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:11:09.232176 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:11:09.232176 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:11:09.232176 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:11:09.232176 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:11:09.232176 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:11:09.233015 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:11:09.233015 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:11:09.233015 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:11:09.233015 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:11:09.233015 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:11:09.233015 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:11:09.233015 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:11:09.233015 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:11:09.233015 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:11:09.233015 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:11:09.233015 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 17 19:11:09.233015 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 17 19:11:09.233015 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 17 19:11:09.233015 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:11:09.233015 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 388/780] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 17 19:11:08 2022] GET /identity/v3/services/network => generated 88 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 17 19:11:09.240814 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba796136-e7d9-4445-8d54-c88d9c866c9c None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:09.357345 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba796136-e7d9-4445-8d54-c88d9c866c9c None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:09.397206 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9389b06-b927-4153-87c5-ef70be5fffae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:09.398233 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9389b06-b927-4153-87c5-ef70be5fffae None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:09.398732 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9389b06-b927-4153-87c5-ef70be5fffae None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:09.399202 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9389b06-b927-4153-87c5-ef70be5fffae None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:09.400197 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9389b06-b927-4153-87c5-ef70be5fffae None admin] RBAC: Adding query filter params (name=network) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:09.400744 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9389b06-b927-4153-87c5-ef70be5fffae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:09.402342 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9389b06-b927-4153-87c5-ef70be5fffae None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:09.411642 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 389/781] 10.222.0.58 () {56 vars in 1112 bytes} [Tue May 17 19:11:09 2022] GET /identity/v3/services?name=network => generated 124 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:09.417887 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9389b06-b927-4153-87c5-ef70be5fffae None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:09.466049 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deb3026-427f-401b-b3e9-049b99cc8942 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:09.517578 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c48510-e964-40c9-bab3-48f6435eb7fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:09.518797 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c48510-e964-40c9-bab3-48f6435eb7fb None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:09.519275 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c48510-e964-40c9-bab3-48f6435eb7fb None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:09.519771 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c48510-e964-40c9-bab3-48f6435eb7fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:09.541896 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c48510-e964-40c9-bab3-48f6435eb7fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:09.543776 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c48510-e964-40c9-bab3-48f6435eb7fb None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:09.553069 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9389b06-b927-4153-87c5-ef70be5fffae None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:09.577435 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-d1c48510-e964-40c9-bab3-48f6435eb7fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:11:09.577435 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:09.577435 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:11:09.577435 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:11:09.577435 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:11:09.577435 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:11:09.577435 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:11:09.577435 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:11:09.577435 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:11:09.577435 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:11:09.577435 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:11:09.577435 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:11:09.577435 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 17 19:11:09.577435 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 17 19:11:09.577435 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 17 19:11:09.578387 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 17 19:11:09.578387 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:11:09.578387 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:11:09.578387 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:11:09.578387 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:11:09.578387 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:11:09.578387 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:11:09.578387 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:11:09.578387 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:11:09.578387 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:11:09.578387 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:11:09.578387 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:11:09.578387 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:11:09.578387 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:11:09.578387 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:11:09.578387 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:11:09.578387 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:11:09.579264 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 17 19:11:09.579264 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 17 19:11:09.579264 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:11:09.579264 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:11:09.579953 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 393/782] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:11:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 372 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:11:09.592439 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c48510-e964-40c9-bab3-48f6435eb7fb None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:09.592586 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae39c05-8f3e-4260-93c1-935bbda5c229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:09.593198 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae39c05-8f3e-4260-93c1-935bbda5c229 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:09.593495 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae39c05-8f3e-4260-93c1-935bbda5c229 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:09.593761 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae39c05-8f3e-4260-93c1-935bbda5c229 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:09.594465 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae39c05-8f3e-4260-93c1-935bbda5c229 None admin] RBAC: Adding query filter params (type=network) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:09.594766 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae39c05-8f3e-4260-93c1-935bbda5c229 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:09.596428 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae39c05-8f3e-4260-93c1-935bbda5c229 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:09.602671 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 390/783] 10.222.0.58 () {56 vars in 1112 bytes} [Tue May 17 19:11:09 2022] GET /identity/v3/services?type=network => generated 350 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:09.608504 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae39c05-8f3e-4260-93c1-935bbda5c229 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:09.706721 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c48510-e964-40c9-bab3-48f6435eb7fb None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:09.734506 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae39c05-8f3e-4260-93c1-935bbda5c229 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:09.741536 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426a7b9a-21c9-409c-97e6-c4f79b2b7217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:09.742192 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426a7b9a-21c9-409c-97e6-c4f79b2b7217 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:09.742466 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426a7b9a-21c9-409c-97e6-c4f79b2b7217 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:09.742736 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426a7b9a-21c9-409c-97e6-c4f79b2b7217 None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:09.743451 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426a7b9a-21c9-409c-97e6-c4f79b2b7217 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:09.743868 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426a7b9a-21c9-409c-97e6-c4f79b2b7217 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:09.745226 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426a7b9a-21c9-409c-97e6-c4f79b2b7217 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:09.758208 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 394/784] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 17 19:11:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:09.759703 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb35a63-2857-4551-97ca-52dceb2ff3e8 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:09.759984 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb35a63-2857-4551-97ca-52dceb2ff3e8 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:09.760263 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb35a63-2857-4551-97ca-52dceb2ff3e8 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:09.761775 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 395/785] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:11:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:11:09.763142 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb35a63-2857-4551-97ca-52dceb2ff3e8 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:09.786081 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3556a3c-cb75-4e8a-84f8-adf1bf1fcbbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:09.786886 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3556a3c-cb75-4e8a-84f8-adf1bf1fcbbd None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:09.787255 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3556a3c-cb75-4e8a-84f8-adf1bf1fcbbd None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:09.787697 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3556a3c-cb75-4e8a-84f8-adf1bf1fcbbd None admin] PATH_INFO: `/v3/endpoints` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:09.788634 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3556a3c-cb75-4e8a-84f8-adf1bf1fcbbd None admin] RBAC: Adding query filter params (interface=public, service_id=f0adbcbf265049d1947c57e9f7cab1f7, region_id=RegionOne) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:09.789035 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3556a3c-cb75-4e8a-84f8-adf1bf1fcbbd None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:09.790495 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3556a3c-cb75-4e8a-84f8-adf1bf1fcbbd None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:09.799918 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 391/786] 10.222.0.58 () {56 vars in 1251 bytes} [Tue May 17 19:11:09 2022] GET /identity/v3/endpoints?service_id=f0adbcbf265049d1947c57e9f7cab1f7&interface=public®ion_id=RegionOne => generated 194 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:09.801921 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be13bbf2-607e-4631-bd34-b8ec9acb1bb1 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:09.802301 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be13bbf2-607e-4631-bd34-b8ec9acb1bb1 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:09.802671 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be13bbf2-607e-4631-bd34-b8ec9acb1bb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:09.820510 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-be13bbf2-607e-4631-bd34-b8ec9acb1bb1 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:11:09.821855 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-be13bbf2-607e-4631-bd34-b8ec9acb1bb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:11:09.887873 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb35a63-2857-4551-97ca-52dceb2ff3e8 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:09.899817 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 392/787] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:11:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 17 19:11:09.900801 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be13bbf2-607e-4631-bd34-b8ec9acb1bb1 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:09.929055 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5998a04-1a61-41cd-ae0b-4a61c57352ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:09.929875 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5998a04-1a61-41cd-ae0b-4a61c57352ab None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:09.930149 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5998a04-1a61-41cd-ae0b-4a61c57352ab None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:09.930414 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5998a04-1a61-41cd-ae0b-4a61c57352ab None admin] PATH_INFO: `/v3/users/swift` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:09.941813 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5998a04-1a61-41cd-ae0b-4a61c57352ab None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:09.945513 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5998a04-1a61-41cd-ae0b-4a61c57352ab None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:09.965857 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-c5998a04-1a61-41cd-ae0b-4a61c57352ab None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 17 19:11:09.965857 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:09.965857 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:11:09.965857 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:11:09.965857 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:11:09.965857 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:11:09.965857 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:11:09.965857 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:11:09.965857 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:11:09.965857 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:11:09.965857 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:11:09.965857 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:11:09.965857 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 17 19:11:09.965857 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_user(user_id) May 17 19:11:09.965857 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 17 19:11:09.965857 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 17 19:11:09.966829 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:11:09.966829 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:11:09.966829 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 17 19:11:09.966829 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 17 19:11:09.966829 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 17 19:11:09.966829 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 17 19:11:09.966829 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:11:09.966829 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:11:09.966829 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:11:09.966829 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:11:09.966829 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:11:09.966829 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:11:09.966829 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:11:09.966829 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:11:09.966829 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:11:09.966829 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:11:09.966829 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:11:09.966829 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:11:09.967906 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:11:09.967906 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:11:09.967906 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 17 19:11:09.967906 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 17 19:11:09.967906 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 17 19:11:09.967906 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 17 19:11:09.967906 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 17 19:11:09.967906 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 17 19:11:09.967906 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 17 19:11:09.967906 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:11:09.974259 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 396/788] 10.222.0.58 () {56 vars in 1096 bytes} [Tue May 17 19:11:09 2022] GET /identity/v3/users/swift => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:11:09.974259 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5998a04-1a61-41cd-ae0b-4a61c57352ab None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:10.026862 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be13bbf2-607e-4631-bd34-b8ec9acb1bb1 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:10.065004 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5d73a7-dd3b-49ca-8073-0d10eb3fcc97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:10.066096 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5d73a7-dd3b-49ca-8073-0d10eb3fcc97 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:10.066580 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5d73a7-dd3b-49ca-8073-0d10eb3fcc97 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:10.067071 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5d73a7-dd3b-49ca-8073-0d10eb3fcc97 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:10.067973 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5d73a7-dd3b-49ca-8073-0d10eb3fcc97 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:10.070042 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5d73a7-dd3b-49ca-8073-0d10eb3fcc97 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:10.077664 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 393/789] 10.222.0.58 () {56 vars in 1087 bytes} [Tue May 17 19:11:09 2022] GET /identity/v3/services => generated 772 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:10.079004 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5d73a7-dd3b-49ca-8073-0d10eb3fcc97 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:10.093372 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5998a04-1a61-41cd-ae0b-4a61c57352ab None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:10.199424 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5d73a7-dd3b-49ca-8073-0d10eb3fcc97 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:10.241025 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694045db-947c-44a7-8b53-6efc47116386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:10.242054 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694045db-947c-44a7-8b53-6efc47116386 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:10.242974 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694045db-947c-44a7-8b53-6efc47116386 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:10.243453 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694045db-947c-44a7-8b53-6efc47116386 None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:10.244779 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694045db-947c-44a7-8b53-6efc47116386 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:10.245424 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694045db-947c-44a7-8b53-6efc47116386 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:10.247688 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694045db-947c-44a7-8b53-6efc47116386 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:10.257237 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64442356-1db8-4803-ab78-b300cca85aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:10.258158 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64442356-1db8-4803-ab78-b300cca85aa3 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:10.258498 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64442356-1db8-4803-ab78-b300cca85aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:10.258841 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64442356-1db8-4803-ab78-b300cca85aa3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:10.271683 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64442356-1db8-4803-ab78-b300cca85aa3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 17 19:11:10.275294 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 394/790] 10.222.0.58 () {56 vars in 1135 bytes} [Tue May 17 19:11:10 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:10.280286 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-64442356-1db8-4803-ab78-b300cca85aa3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 17 19:11:10.280286 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:10.280286 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:11:10.280286 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:11:10.280286 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:11:10.280286 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:11:10.280286 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:11:10.280286 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:11:10.280286 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:11:10.280286 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:11:10.280286 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:11:10.280286 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:11:10.280286 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 17 19:11:10.280286 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) May 17 19:11:10.280286 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 17 19:11:10.280286 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 17 19:11:10.281280 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 17 19:11:10.281280 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 17 19:11:10.281280 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 17 19:11:10.281280 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 17 19:11:10.281280 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:11:10.281280 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:11:10.281280 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:11:10.281280 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:11:10.281280 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:11:10.281280 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:11:10.281280 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:11:10.281280 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:11:10.281280 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:11:10.281280 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:11:10.281280 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:11:10.281280 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:11:10.282127 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:11:10.282127 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:11:10.282127 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:11:10.282127 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:11:10.282127 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 17 19:11:10.282127 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 17 19:11:10.282127 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 17 19:11:10.282127 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 17 19:11:10.282127 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 17 19:11:10.282127 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 17 19:11:10.282127 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 17 19:11:10.282127 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:11:10.283878 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694045db-947c-44a7-8b53-6efc47116386 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:10.286973 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 397/791] 10.222.0.58 () {56 vars in 1102 bytes} [Tue May 17 19:11:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:11:10.288863 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64442356-1db8-4803-ab78-b300cca85aa3 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:10.410052 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694045db-947c-44a7-8b53-6efc47116386 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:10.435411 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64442356-1db8-4803-ab78-b300cca85aa3 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:10.455788 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07745b5b-06c6-4ff7-9a67-de92588a6f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:10.456837 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07745b5b-06c6-4ff7-9a67-de92588a6f6c None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:10.457715 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07745b5b-06c6-4ff7-9a67-de92588a6f6c None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:10.458142 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07745b5b-06c6-4ff7-9a67-de92588a6f6c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:10.472142 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07745b5b-06c6-4ff7-9a67-de92588a6f6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:10.472434 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe309f7-ec8f-4d31-960b-f73096d5f2d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:10.473099 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe309f7-ec8f-4d31-960b-f73096d5f2d4 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:10.473375 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe309f7-ec8f-4d31-960b-f73096d5f2d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:10.473647 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe309f7-ec8f-4d31-960b-f73096d5f2d4 None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:10.474388 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe309f7-ec8f-4d31-960b-f73096d5f2d4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:10.474689 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe309f7-ec8f-4d31-960b-f73096d5f2d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:10.475099 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07745b5b-06c6-4ff7-9a67-de92588a6f6c None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:10.476845 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe309f7-ec8f-4d31-960b-f73096d5f2d4 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:10.485369 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 398/792] 10.222.0.58 () {56 vars in 1103 bytes} [Tue May 17 19:11:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:10.496021 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-07745b5b-06c6-4ff7-9a67-de92588a6f6c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:11:10.496021 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:10.496021 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:11:10.496021 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:11:10.496021 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:11:10.496021 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:11:10.496021 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:11:10.496021 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:11:10.496021 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:11:10.496021 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:11:10.496021 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:11:10.496021 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:11:10.496021 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 17 19:11:10.496021 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 17 19:11:10.496021 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 17 19:11:10.496913 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 17 19:11:10.496913 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:11:10.496913 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:11:10.496913 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:11:10.496913 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:11:10.496913 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:11:10.496913 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:11:10.496913 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:11:10.496913 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:11:10.496913 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:11:10.496913 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:11:10.496913 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:11:10.496913 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:11:10.496913 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:11:10.496913 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:11:10.496913 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:11:10.496913 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:11:10.497824 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 17 19:11:10.497824 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 17 19:11:10.497824 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:11:10.497824 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:11:10.497824 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 395/793] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:11:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 17 19:11:10.498601 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe309f7-ec8f-4d31-960b-f73096d5f2d4 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:10.502614 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07745b5b-06c6-4ff7-9a67-de92588a6f6c None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:10.630348 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe309f7-ec8f-4d31-960b-f73096d5f2d4 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:10.632781 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07745b5b-06c6-4ff7-9a67-de92588a6f6c None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:10.664534 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9164a5-b062-47a8-a051-0b545000259c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:10.665420 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9164a5-b062-47a8-a051-0b545000259c None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:10.665764 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9164a5-b062-47a8-a051-0b545000259c None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:10.666104 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9164a5-b062-47a8-a051-0b545000259c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:10.672123 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58cfe61-2d9b-4ecd-8853-2c4f59174cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:10.672778 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58cfe61-2d9b-4ecd-8853-2c4f59174cef None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:10.673054 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58cfe61-2d9b-4ecd-8853-2c4f59174cef None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:10.673745 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58cfe61-2d9b-4ecd-8853-2c4f59174cef None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:10.674628 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58cfe61-2d9b-4ecd-8853-2c4f59174cef None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:10.676049 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58cfe61-2d9b-4ecd-8853-2c4f59174cef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:10.678814 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58cfe61-2d9b-4ecd-8853-2c4f59174cef None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:10.681201 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9164a5-b062-47a8-a051-0b545000259c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:10.682732 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9164a5-b062-47a8-a051-0b545000259c None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:10.694176 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 396/794] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 17 19:11:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:11:10.695790 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5702fa09-d075-4031-abd8-0f98bc2cbbda None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:10.696295 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5702fa09-d075-4031-abd8-0f98bc2cbbda None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:10.696817 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5702fa09-d075-4031-abd8-0f98bc2cbbda None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:10.702288 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 397/795] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:11:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:11:10.704149 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0996584a-2f14-4845-9e30-3864876819fa None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:10.704633 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0996584a-2f14-4845-9e30-3864876819fa None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:10.705100 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0996584a-2f14-4845-9e30-3864876819fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:10.705633 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-ea9164a5-b062-47a8-a051-0b545000259c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:11:10.705633 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:10.705633 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:11:10.705633 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:11:10.705633 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:11:10.705633 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:11:10.705633 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:11:10.705633 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:11:10.705633 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:11:10.705633 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:11:10.705633 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:11:10.705633 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:11:10.705633 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 17 19:11:10.705633 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 17 19:11:10.705633 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 17 19:11:10.706585 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 17 19:11:10.706585 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:11:10.706585 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:11:10.706585 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:11:10.706585 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:11:10.706585 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:11:10.706585 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:11:10.706585 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:11:10.706585 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:11:10.706585 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:11:10.706585 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:11:10.706585 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:11:10.706585 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:11:10.706585 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:11:10.706585 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:11:10.706585 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:11:10.706585 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:11:10.707855 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 17 19:11:10.707855 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 17 19:11:10.707855 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:11:10.707855 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:11:10.707855 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 399/796] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:11:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:11:10.709793 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9164a5-b062-47a8-a051-0b545000259c None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:10.727343 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-0996584a-2f14-4845-9e30-3864876819fa None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:11:10.729441 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-0996584a-2f14-4845-9e30-3864876819fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:11:10.807726 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 398/797] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:11:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 17 19:11:10.808948 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0996584a-2f14-4845-9e30-3864876819fa None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:10.815316 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9164a5-b062-47a8-a051-0b545000259c None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:10.849797 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28344789-2e98-4f21-9644-e7aec7cf341d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:10.850614 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28344789-2e98-4f21-9644-e7aec7cf341d None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:10.850948 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28344789-2e98-4f21-9644-e7aec7cf341d None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:10.851207 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28344789-2e98-4f21-9644-e7aec7cf341d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:10.858375 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28344789-2e98-4f21-9644-e7aec7cf341d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:10.860452 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28344789-2e98-4f21-9644-e7aec7cf341d None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:10.879460 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-28344789-2e98-4f21-9644-e7aec7cf341d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 17 19:11:10.879460 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:10.879460 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:11:10.879460 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:11:10.879460 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:11:10.879460 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:11:10.879460 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:11:10.879460 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:11:10.879460 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:11:10.879460 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:11:10.879460 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:11:10.879460 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:11:10.879460 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 17 19:11:10.879460 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(project_id) May 17 19:11:10.879460 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 17 19:11:10.880550 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 17 19:11:10.880550 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:11:10.880550 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:11:10.880550 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:11:10.880550 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:11:10.880550 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:11:10.880550 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:11:10.880550 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:11:10.880550 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:11:10.880550 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:11:10.880550 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:11:10.880550 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:11:10.880550 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:11:10.880550 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:11:10.880550 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:11:10.880550 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:11:10.880550 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:11:10.881519 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 17 19:11:10.881519 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 17 19:11:10.881519 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 17 19:11:10.881519 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 17 19:11:10.881519 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 17 19:11:10.881519 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 17 19:11:10.881519 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 17 19:11:10.881519 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:11:10.882086 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 400/798] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 17 19:11:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:11:10.887660 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28344789-2e98-4f21-9644-e7aec7cf341d None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:10.929587 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0996584a-2f14-4845-9e30-3864876819fa None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:10.967912 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ef4b68-508b-4c5c-9c7c-a319e0a69114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:10.968869 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ef4b68-508b-4c5c-9c7c-a319e0a69114 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:10.969333 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ef4b68-508b-4c5c-9c7c-a319e0a69114 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:10.969773 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ef4b68-508b-4c5c-9c7c-a319e0a69114 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:10.970651 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ef4b68-508b-4c5c-9c7c-a319e0a69114 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:10.971402 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ef4b68-508b-4c5c-9c7c-a319e0a69114 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:10.973027 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ef4b68-508b-4c5c-9c7c-a319e0a69114 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:10.985359 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 399/799] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 17 19:11:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:10.986536 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ef4b68-508b-4c5c-9c7c-a319e0a69114 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:10.994664 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28344789-2e98-4f21-9644-e7aec7cf341d None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:11.035169 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461e639a-ac2d-41d8-a7b5-fcf881db0f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:11.036389 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461e639a-ac2d-41d8-a7b5-fcf881db0f4e None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:11.036877 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461e639a-ac2d-41d8-a7b5-fcf881db0f4e None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:11.037337 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461e639a-ac2d-41d8-a7b5-fcf881db0f4e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:11.051575 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461e639a-ac2d-41d8-a7b5-fcf881db0f4e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 17 19:11:11.062156 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-461e639a-ac2d-41d8-a7b5-fcf881db0f4e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 17 19:11:11.062156 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:11.062156 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:11:11.062156 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:11:11.062156 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:11:11.062156 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:11:11.062156 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:11:11.062156 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:11:11.062156 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:11:11.062156 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:11:11.062156 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:11:11.062156 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:11:11.062156 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 17 19:11:11.062156 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) May 17 19:11:11.062156 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 17 19:11:11.062156 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 17 19:11:11.063098 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 17 19:11:11.063098 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 17 19:11:11.063098 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 17 19:11:11.063098 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 17 19:11:11.063098 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:11:11.063098 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:11:11.063098 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:11:11.063098 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:11:11.063098 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:11:11.063098 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:11:11.063098 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:11:11.063098 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:11:11.063098 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:11:11.063098 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:11:11.063098 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:11:11.063098 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:11:11.064000 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:11:11.064000 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:11:11.064000 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:11:11.064000 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:11:11.064000 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 17 19:11:11.064000 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 17 19:11:11.064000 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 17 19:11:11.064000 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 17 19:11:11.064000 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 17 19:11:11.064000 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 17 19:11:11.064000 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 17 19:11:11.064000 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:11:11.067050 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 401/800] 10.222.0.58 () {56 vars in 1096 bytes} [Tue May 17 19:11:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:11:11.067050 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461e639a-ac2d-41d8-a7b5-fcf881db0f4e None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:11.114420 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ef4b68-508b-4c5c-9c7c-a319e0a69114 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:11.155682 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca9dc62-0792-41e4-a7cc-227d99ea063c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:11.156354 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca9dc62-0792-41e4-a7cc-227d99ea063c None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:11.156638 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca9dc62-0792-41e4-a7cc-227d99ea063c None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:11.156910 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca9dc62-0792-41e4-a7cc-227d99ea063c None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:11.157692 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca9dc62-0792-41e4-a7cc-227d99ea063c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:11.158001 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca9dc62-0792-41e4-a7cc-227d99ea063c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:11.160052 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca9dc62-0792-41e4-a7cc-227d99ea063c None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:11.175567 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 400/801] 10.222.0.58 () {56 vars in 1148 bytes} [Tue May 17 19:11:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:11.176708 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca9dc62-0792-41e4-a7cc-227d99ea063c None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:11.192830 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461e639a-ac2d-41d8-a7b5-fcf881db0f4e None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:11.236191 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52e78d9-04e5-4991-9344-4a23dc045202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:11.237464 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52e78d9-04e5-4991-9344-4a23dc045202 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:11.237943 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52e78d9-04e5-4991-9344-4a23dc045202 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:11.238406 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52e78d9-04e5-4991-9344-4a23dc045202 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:11.247328 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52e78d9-04e5-4991-9344-4a23dc045202 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:11.249204 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52e78d9-04e5-4991-9344-4a23dc045202 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:11.266135 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-e52e78d9-04e5-4991-9344-4a23dc045202 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 17 19:11:11.266135 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:11.266135 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:11:11.266135 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:11:11.266135 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:11:11.266135 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:11:11.266135 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:11:11.266135 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:11:11.266135 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:11:11.266135 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:11:11.266135 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:11:11.266135 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:11:11.266135 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 17 19:11:11.266135 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_user(user_id) May 17 19:11:11.266135 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 17 19:11:11.266135 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 17 19:11:11.267144 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:11:11.267144 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:11:11.267144 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 17 19:11:11.267144 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 17 19:11:11.267144 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 17 19:11:11.267144 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 17 19:11:11.267144 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:11:11.267144 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:11:11.267144 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:11:11.267144 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:11:11.267144 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:11:11.267144 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:11:11.267144 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:11:11.267144 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:11:11.267144 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:11:11.267144 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:11:11.267144 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:11:11.267144 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:11:11.268149 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:11:11.268149 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:11:11.268149 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 17 19:11:11.268149 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 17 19:11:11.268149 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 17 19:11:11.268149 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 17 19:11:11.268149 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 17 19:11:11.268149 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 17 19:11:11.268149 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 17 19:11:11.268149 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:11:11.275612 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 402/802] 10.222.0.58 () {56 vars in 1117 bytes} [Tue May 17 19:11:11 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:11:11.276625 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52e78d9-04e5-4991-9344-4a23dc045202 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:11.301081 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca9dc62-0792-41e4-a7cc-227d99ea063c None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:11.345596 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f951f7-1404-437b-add7-8d07da7e6359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:11.346531 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f951f7-1404-437b-add7-8d07da7e6359 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:11.347384 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f951f7-1404-437b-add7-8d07da7e6359 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:11.347680 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f951f7-1404-437b-add7-8d07da7e6359 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:11.348438 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f951f7-1404-437b-add7-8d07da7e6359 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:11.348743 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f951f7-1404-437b-add7-8d07da7e6359 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:11.350134 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f951f7-1404-437b-add7-8d07da7e6359 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:11.366076 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 401/803] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 17 19:11:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:11.366076 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f951f7-1404-437b-add7-8d07da7e6359 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:11.382670 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52e78d9-04e5-4991-9344-4a23dc045202 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:11.418638 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f793ef-fd3a-49ae-902a-a52261d565f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:11.419891 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f793ef-fd3a-49ae-902a-a52261d565f6 None admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:11.420384 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f793ef-fd3a-49ae-902a-a52261d565f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:11.420860 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f793ef-fd3a-49ae-902a-a52261d565f6 None admin] PATH_INFO: `/v3/projects/1a9d3a0631284c9fb113266036993e2d/users/b0cb292002434d30bc81f370d120a2a2/roles/60af5088793e4379b88821a86552a344` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:11.423197 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f793ef-fd3a-49ae-902a-a52261d565f6 None admin] RBAC: Authorizing `identity:create_grant(project_id=1a9d3a0631284c9fb113266036993e2d, user_id=b0cb292002434d30bc81f370d120a2a2, role_id=60af5088793e4379b88821a86552a344)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:11.425005 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f793ef-fd3a-49ae-902a-a52261d565f6 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:11.474075 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f951f7-1404-437b-add7-8d07da7e6359 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:11.506215 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210a8d40-f8e8-45b8-91a5-bcf230a07658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:11.506804 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210a8d40-f8e8-45b8-91a5-bcf230a07658 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:11.507079 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210a8d40-f8e8-45b8-91a5-bcf230a07658 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:11.507348 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210a8d40-f8e8-45b8-91a5-bcf230a07658 None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:11.508268 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210a8d40-f8e8-45b8-91a5-bcf230a07658 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:11.508575 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210a8d40-f8e8-45b8-91a5-bcf230a07658 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:11.515792 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 403/804] 10.222.0.58 () {58 vars in 1439 bytes} [Tue May 17 19:11:11 2022] PUT /identity/v3/projects/1a9d3a0631284c9fb113266036993e2d/users/b0cb292002434d30bc81f370d120a2a2/roles/60af5088793e4379b88821a86552a344 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:11:11.516305 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210a8d40-f8e8-45b8-91a5-bcf230a07658 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:11.516412 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f793ef-fd3a-49ae-902a-a52261d565f6 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:11.532907 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 402/805] 10.222.0.58 () {56 vars in 1149 bytes} [Tue May 17 19:11:11 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:11.534367 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e4a5d0-7730-4426-a893-299cdb6ad91e None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:11.534658 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e4a5d0-7730-4426-a893-299cdb6ad91e None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:11.534932 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e4a5d0-7730-4426-a893-299cdb6ad91e None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:11.540659 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 403/806] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:11:11 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:11:11.541600 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e4a5d0-7730-4426-a893-299cdb6ad91e None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:11.649372 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e4a5d0-7730-4426-a893-299cdb6ad91e None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:11.649845 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f793ef-fd3a-49ae-902a-a52261d565f6 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:11.690289 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083b2e99-62cc-4cf8-827d-e5ed7b7fbc36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:11.691712 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083b2e99-62cc-4cf8-827d-e5ed7b7fbc36 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:11.691995 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083b2e99-62cc-4cf8-827d-e5ed7b7fbc36 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:11.692320 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083b2e99-62cc-4cf8-827d-e5ed7b7fbc36 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:11.693852 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa935c3d-be51-4258-9547-1c875820c96e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:11.694867 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa935c3d-be51-4258-9547-1c875820c96e None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:11.695578 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa935c3d-be51-4258-9547-1c875820c96e None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:11.695858 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa935c3d-be51-4258-9547-1c875820c96e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:11.704248 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083b2e99-62cc-4cf8-827d-e5ed7b7fbc36 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:11.710304 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083b2e99-62cc-4cf8-827d-e5ed7b7fbc36 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:11.712946 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa935c3d-be51-4258-9547-1c875820c96e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:11.714950 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa935c3d-be51-4258-9547-1c875820c96e None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:11.731953 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-083b2e99-62cc-4cf8-827d-e5ed7b7fbc36 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:11:11.731953 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:11.731953 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:11:11.731953 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:11:11.731953 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:11:11.731953 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:11:11.731953 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:11:11.731953 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:11:11.731953 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:11:11.731953 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:11:11.731953 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:11:11.731953 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:11:11.731953 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 17 19:11:11.731953 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 17 19:11:11.731953 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 17 19:11:11.732885 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 17 19:11:11.732885 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:11:11.732885 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:11:11.732885 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:11:11.732885 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:11:11.732885 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:11:11.732885 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:11:11.732885 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:11:11.732885 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:11:11.732885 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:11:11.732885 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:11:11.732885 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:11:11.732885 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:11:11.732885 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:11:11.732885 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:11:11.732885 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:11:11.732885 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:11:11.733795 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 17 19:11:11.733795 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 17 19:11:11.733795 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:11:11.733795 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:11:11.733795 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 404/807] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:11:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:11:11.735858 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf9dbe5-076f-4e2a-bb37-2ca9a52ef8f1 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:11.735858 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf9dbe5-076f-4e2a-bb37-2ca9a52ef8f1 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:11.735858 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf9dbe5-076f-4e2a-bb37-2ca9a52ef8f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:11.736715 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-fa935c3d-be51-4258-9547-1c875820c96e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:11:11.736715 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:11.736715 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:11:11.736715 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:11:11.736715 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:11:11.736715 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:11:11.736715 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:11:11.736715 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:11:11.736715 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:11:11.736715 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:11:11.736715 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:11:11.736715 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:11:11.736715 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 17 19:11:11.736715 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 17 19:11:11.736715 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 17 19:11:11.739199 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 17 19:11:11.739199 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:11:11.739199 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:11:11.739199 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:11:11.739199 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:11:11.739199 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:11:11.739199 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:11:11.739199 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:11:11.739199 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:11:11.739199 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:11:11.739199 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:11:11.739199 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:11:11.739199 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:11:11.739199 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:11:11.739199 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:11:11.739199 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:11:11.739199 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:11:11.741353 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 17 19:11:11.741353 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 17 19:11:11.741353 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:11:11.741353 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:11:11.741353 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 404/808] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:11:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:11:11.745425 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa935c3d-be51-4258-9547-1c875820c96e None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:11.755097 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-cbf9dbe5-076f-4e2a-bb37-2ca9a52ef8f1 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:11:11.757862 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-cbf9dbe5-076f-4e2a-bb37-2ca9a52ef8f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:11:11.841292 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 405/809] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:11:11 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 17 19:11:11.842549 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf9dbe5-076f-4e2a-bb37-2ca9a52ef8f1 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:11.869481 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa935c3d-be51-4258-9547-1c875820c96e None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:11.906127 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210475ad-b948-424f-ae36-e13a53e56c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:11.906793 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210475ad-b948-424f-ae36-e13a53e56c68 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:11.907074 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210475ad-b948-424f-ae36-e13a53e56c68 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:11.907359 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210475ad-b948-424f-ae36-e13a53e56c68 None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:11.908105 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210475ad-b948-424f-ae36-e13a53e56c68 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:11.908410 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210475ad-b948-424f-ae36-e13a53e56c68 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:11.910351 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210475ad-b948-424f-ae36-e13a53e56c68 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:11.922614 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 405/810] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 17 19:11:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:11:11.923533 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210475ad-b948-424f-ae36-e13a53e56c68 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:11.948150 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf9dbe5-076f-4e2a-bb37-2ca9a52ef8f1 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:11.984334 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7908aafb-e68a-4779-b68e-15b77534af8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:11.985403 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7908aafb-e68a-4779-b68e-15b77534af8c None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:11.985888 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7908aafb-e68a-4779-b68e-15b77534af8c None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:11.986361 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7908aafb-e68a-4779-b68e-15b77534af8c None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:11.987285 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7908aafb-e68a-4779-b68e-15b77534af8c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:11.988343 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7908aafb-e68a-4779-b68e-15b77534af8c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:11.989929 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7908aafb-e68a-4779-b68e-15b77534af8c None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:12.001966 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 406/811] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 17 19:11:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:12.007359 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7908aafb-e68a-4779-b68e-15b77534af8c None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:12.024255 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210475ad-b948-424f-ae36-e13a53e56c68 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:12.059733 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb9cdc7-98c9-4f97-a0f6-ae164ae42b50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:12.060558 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb9cdc7-98c9-4f97-a0f6-ae164ae42b50 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:12.060837 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb9cdc7-98c9-4f97-a0f6-ae164ae42b50 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:12.061292 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb9cdc7-98c9-4f97-a0f6-ae164ae42b50 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:12.061748 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb9cdc7-98c9-4f97-a0f6-ae164ae42b50 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:12.063124 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb9cdc7-98c9-4f97-a0f6-ae164ae42b50 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:12.075409 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-2eb9cdc7-98c9-4f97-a0f6-ae164ae42b50 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 17 19:11:12.075409 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:12.075409 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 17 19:11:12.075409 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 17 19:11:12.075409 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 17 19:11:12.075409 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 17 19:11:12.075409 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 17 19:11:12.075409 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 17 19:11:12.075409 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 17 19:11:12.075409 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:11:12.075409 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 17 19:11:12.075409 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:11:12.075409 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:12.075409 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:11:12.075409 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:11:12.075409 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:11:12.076390 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:11:12.076390 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:11:12.076390 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:11:12.076390 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:11:12.076390 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:11:12.076390 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:11:12.076390 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:11:12.076390 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 17 19:11:12.076390 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_service(service_id) May 17 19:11:12.076390 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 17 19:11:12.076390 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 17 19:11:12.076390 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:11:12.076390 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:11:12.076390 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:11:12.076390 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:11:12.076390 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:11:12.077237 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:11:12.077237 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:11:12.077237 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:11:12.077237 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:11:12.077237 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:11:12.077237 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:11:12.077237 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:11:12.077237 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:11:12.077237 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:11:12.077237 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:11:12.077237 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:11:12.077237 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 17 19:11:12.077237 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 17 19:11:12.077237 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 17 19:11:12.077237 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:11:12.082259 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 406/812] 10.222.0.58 () {56 vars in 1129 bytes} [Tue May 17 19:11:11 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:11:12.083288 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb9cdc7-98c9-4f97-a0f6-ae164ae42b50 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:12.123120 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7908aafb-e68a-4779-b68e-15b77534af8c None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:12.165558 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a996f6-5fc0-4b95-9478-e05a91e64c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:12.166670 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a996f6-5fc0-4b95-9478-e05a91e64c0c None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:12.166951 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a996f6-5fc0-4b95-9478-e05a91e64c0c None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:12.167223 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a996f6-5fc0-4b95-9478-e05a91e64c0c None admin] PATH_INFO: `/v3/users/nova` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:12.176627 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a996f6-5fc0-4b95-9478-e05a91e64c0c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:12.178797 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a996f6-5fc0-4b95-9478-e05a91e64c0c None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:12.189632 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb9cdc7-98c9-4f97-a0f6-ae164ae42b50 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:12.196799 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-59a996f6-5fc0-4b95-9478-e05a91e64c0c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 17 19:11:12.196799 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:12.196799 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:11:12.196799 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:11:12.196799 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:11:12.196799 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:11:12.196799 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:11:12.196799 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:11:12.196799 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:11:12.196799 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:11:12.196799 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:11:12.196799 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:11:12.196799 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 17 19:11:12.196799 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_user(user_id) May 17 19:11:12.196799 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 17 19:11:12.196799 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 17 19:11:12.197770 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:11:12.197770 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:11:12.197770 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 17 19:11:12.197770 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 17 19:11:12.197770 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 17 19:11:12.197770 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 17 19:11:12.197770 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:11:12.197770 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:11:12.197770 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:11:12.197770 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:11:12.197770 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:11:12.197770 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:11:12.197770 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:11:12.197770 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:11:12.197770 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:11:12.197770 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:11:12.197770 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:11:12.197770 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:11:12.198743 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:11:12.198743 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:11:12.198743 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 17 19:11:12.198743 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 17 19:11:12.198743 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 17 19:11:12.198743 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 17 19:11:12.198743 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 17 19:11:12.198743 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 17 19:11:12.198743 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 17 19:11:12.198743 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:11:12.198743 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 407/813] 10.222.0.58 () {56 vars in 1093 bytes} [Tue May 17 19:11:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:11:12.199341 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a996f6-5fc0-4b95-9478-e05a91e64c0c None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:12.233978 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca030168-dd5e-4cd7-bb75-1746ba5119d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:12.235112 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca030168-dd5e-4cd7-bb75-1746ba5119d7 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:12.235394 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca030168-dd5e-4cd7-bb75-1746ba5119d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:12.237792 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca030168-dd5e-4cd7-bb75-1746ba5119d7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:12.245849 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca030168-dd5e-4cd7-bb75-1746ba5119d7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:12.247303 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca030168-dd5e-4cd7-bb75-1746ba5119d7 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:12.264848 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-ca030168-dd5e-4cd7-bb75-1746ba5119d7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 17 19:11:12.264848 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:12.264848 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:11:12.264848 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:11:12.264848 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:11:12.264848 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:11:12.264848 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:11:12.264848 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:11:12.264848 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:11:12.264848 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:11:12.264848 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:11:12.264848 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:11:12.264848 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 17 19:11:12.264848 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(project_id) May 17 19:11:12.264848 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 17 19:11:12.265799 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 17 19:11:12.265799 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:11:12.265799 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:11:12.265799 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:11:12.265799 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:11:12.265799 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:11:12.265799 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:11:12.265799 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:11:12.265799 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:11:12.265799 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:11:12.265799 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:11:12.265799 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:11:12.265799 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:11:12.265799 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:11:12.265799 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:11:12.265799 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:11:12.265799 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:11:12.266713 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 17 19:11:12.266713 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 17 19:11:12.266713 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 17 19:11:12.266713 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 17 19:11:12.266713 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 17 19:11:12.266713 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 17 19:11:12.266713 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 17 19:11:12.266713 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:11:12.274202 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 407/814] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 17 19:11:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:11:12.275468 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca030168-dd5e-4cd7-bb75-1746ba5119d7 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:12.324813 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a996f6-5fc0-4b95-9478-e05a91e64c0c None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:12.360833 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f7537a-71cb-42f2-8639-98b43ae2ea02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:12.361905 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f7537a-71cb-42f2-8639-98b43ae2ea02 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:12.362485 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f7537a-71cb-42f2-8639-98b43ae2ea02 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:12.362981 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f7537a-71cb-42f2-8639-98b43ae2ea02 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:12.364003 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f7537a-71cb-42f2-8639-98b43ae2ea02 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:12.364534 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f7537a-71cb-42f2-8639-98b43ae2ea02 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:12.368067 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f7537a-71cb-42f2-8639-98b43ae2ea02 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:12.376948 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 408/815] 10.222.0.58 () {56 vars in 1124 bytes} [Tue May 17 19:11:12 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:12.378786 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f7537a-71cb-42f2-8639-98b43ae2ea02 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:12.392313 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca030168-dd5e-4cd7-bb75-1746ba5119d7 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:12.428883 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb150cce-8a6a-4e64-a676-b678f46a4128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:12.429559 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb150cce-8a6a-4e64-a676-b678f46a4128 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:12.429840 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb150cce-8a6a-4e64-a676-b678f46a4128 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:12.430113 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb150cce-8a6a-4e64-a676-b678f46a4128 None admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:12.430979 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb150cce-8a6a-4e64-a676-b678f46a4128 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:12.431677 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb150cce-8a6a-4e64-a676-b678f46a4128 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:12.433440 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb150cce-8a6a-4e64-a676-b678f46a4128 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:12.456922 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 408/816] 10.222.0.58 () {56 vars in 1133 bytes} [Tue May 17 19:11:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:12.457785 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb150cce-8a6a-4e64-a676-b678f46a4128 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:12.492165 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f7537a-71cb-42f2-8639-98b43ae2ea02 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:12.527278 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2f5767-53eb-477f-9d4e-41cf85647344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:12.528286 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2f5767-53eb-477f-9d4e-41cf85647344 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:12.528794 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2f5767-53eb-477f-9d4e-41cf85647344 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:12.529267 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2f5767-53eb-477f-9d4e-41cf85647344 None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:12.530246 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2f5767-53eb-477f-9d4e-41cf85647344 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:12.530770 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2f5767-53eb-477f-9d4e-41cf85647344 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:12.532608 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2f5767-53eb-477f-9d4e-41cf85647344 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:12.551157 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 409/817] 10.222.0.58 () {56 vars in 1148 bytes} [Tue May 17 19:11:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:12.552091 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2f5767-53eb-477f-9d4e-41cf85647344 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:12.564143 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb150cce-8a6a-4e64-a676-b678f46a4128 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:12.598131 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d650ef92-f403-467d-8593-0e4b9cf906e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:12.598858 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d650ef92-f403-467d-8593-0e4b9cf906e3 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:12.599167 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d650ef92-f403-467d-8593-0e4b9cf906e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:12.599443 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d650ef92-f403-467d-8593-0e4b9cf906e3 None admin] PATH_INFO: `/v3/services` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:12.600167 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d650ef92-f403-467d-8593-0e4b9cf906e3 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:12.600467 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d650ef92-f403-467d-8593-0e4b9cf906e3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:12.601822 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d650ef92-f403-467d-8593-0e4b9cf906e3 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:12.606551 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 409/818] 10.222.0.58 () {56 vars in 1124 bytes} [Tue May 17 19:11:12 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:12.607811 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184b0518-09f4-449e-a05b-03bf3a55d4af None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:12.608096 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184b0518-09f4-449e-a05b-03bf3a55d4af None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:12.608370 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184b0518-09f4-449e-a05b-03bf3a55d4af None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:12.611602 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 410/819] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:11:12 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:11:12.611602 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184b0518-09f4-449e-a05b-03bf3a55d4af None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:12.654042 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2f5767-53eb-477f-9d4e-41cf85647344 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:12.689542 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba732b2-2ac4-41ba-b707-8cdc5dd30328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:12.690246 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba732b2-2ac4-41ba-b707-8cdc5dd30328 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:12.690585 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba732b2-2ac4-41ba-b707-8cdc5dd30328 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:12.690862 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba732b2-2ac4-41ba-b707-8cdc5dd30328 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:12.710447 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184b0518-09f4-449e-a05b-03bf3a55d4af None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:12.745734 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2244f7-bad9-45b3-9554-ef183a46044e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:12.746614 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2244f7-bad9-45b3-9554-ef183a46044e None admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:12.746896 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2244f7-bad9-45b3-9554-ef183a46044e None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:12.747171 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2244f7-bad9-45b3-9554-ef183a46044e None admin] PATH_INFO: `/v3/projects/1a9d3a0631284c9fb113266036993e2d/users/2b90b57c542440448579b3aadda8b241/roles/1754a95b778c4e20b53b20cb52ada92b` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:12.761786 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2244f7-bad9-45b3-9554-ef183a46044e None admin] RBAC: Authorizing `identity:create_grant(project_id=1a9d3a0631284c9fb113266036993e2d, user_id=2b90b57c542440448579b3aadda8b241, role_id=1754a95b778c4e20b53b20cb52ada92b)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:12.763228 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2244f7-bad9-45b3-9554-ef183a46044e None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:12.767134 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba732b2-2ac4-41ba-b707-8cdc5dd30328 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:12.769173 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba732b2-2ac4-41ba-b707-8cdc5dd30328 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:12.822654 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 411/820] 10.222.0.58 () {58 vars in 1439 bytes} [Tue May 17 19:11:12 2022] PUT /identity/v3/projects/1a9d3a0631284c9fb113266036993e2d/users/2b90b57c542440448579b3aadda8b241/roles/1754a95b778c4e20b53b20cb52ada92b => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:11:12.823382 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2244f7-bad9-45b3-9554-ef183a46044e None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:12.899807 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 410/821] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 17 19:11:12 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 17 19:11:12.902091 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2af312-727d-4a83-bec2-a36651074775 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:12.902862 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2af312-727d-4a83-bec2-a36651074775 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:12.903319 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2af312-727d-4a83-bec2-a36651074775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:12.925651 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-4c2af312-727d-4a83-bec2-a36651074775 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:11:12.927026 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-4c2af312-727d-4a83-bec2-a36651074775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:11:12.952266 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2244f7-bad9-45b3-9554-ef183a46044e None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:12.989793 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaca23a-4edf-42b2-a927-3097d9ba6232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:12.990714 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaca23a-4edf-42b2-a927-3097d9ba6232 None admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:12.991237 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaca23a-4edf-42b2-a927-3097d9ba6232 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:12.991741 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaca23a-4edf-42b2-a927-3097d9ba6232 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:12.992716 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaca23a-4edf-42b2-a927-3097d9ba6232 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:12.994418 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaca23a-4edf-42b2-a927-3097d9ba6232 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:13.019046 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 411/822] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:11:12 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 17 19:11:13.023623 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2af312-727d-4a83-bec2-a36651074775 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:13.104557 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 412/823] 10.222.0.58 () {60 vars in 1144 bytes} [Tue May 17 19:11:12 2022] POST /identity/v3/endpoints => generated 346 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:11:13.105515 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaca23a-4edf-42b2-a927-3097d9ba6232 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:13.160394 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2af312-727d-4a83-bec2-a36651074775 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:13.239441 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11c070a-cb00-48bb-826e-586cd4416e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:13.240272 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11c070a-cb00-48bb-826e-586cd4416e53 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:13.240545 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11c070a-cb00-48bb-826e-586cd4416e53 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:13.240811 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11c070a-cb00-48bb-826e-586cd4416e53 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:13.261196 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11c070a-cb00-48bb-826e-586cd4416e53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:13.262668 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11c070a-cb00-48bb-826e-586cd4416e53 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:13.293849 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-f11c070a-cb00-48bb-826e-586cd4416e53 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:11:13.293849 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:13.293849 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:11:13.293849 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:11:13.293849 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:11:13.293849 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:11:13.293849 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:11:13.293849 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:11:13.293849 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:11:13.293849 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:11:13.293849 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:11:13.293849 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:11:13.293849 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 17 19:11:13.293849 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 17 19:11:13.293849 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 17 19:11:13.294998 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 17 19:11:13.294998 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:11:13.294998 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:11:13.294998 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:11:13.294998 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:11:13.294998 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:11:13.294998 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:11:13.294998 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:11:13.294998 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:11:13.294998 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:11:13.294998 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:11:13.294998 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:11:13.294998 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:11:13.294998 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:11:13.294998 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:11:13.294998 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:11:13.294998 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:11:13.296198 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 17 19:11:13.296198 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 17 19:11:13.296198 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:11:13.296198 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:11:13.301365 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 412/824] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:11:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:11:13.305559 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11c070a-cb00-48bb-826e-586cd4416e53 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:13.329437 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaca23a-4edf-42b2-a927-3097d9ba6232 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:13.365736 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff5adeb-ab0c-4403-bfef-e076d80d7812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:13.366900 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff5adeb-ab0c-4403-bfef-e076d80d7812 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:13.367395 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff5adeb-ab0c-4403-bfef-e076d80d7812 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:13.367885 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff5adeb-ab0c-4403-bfef-e076d80d7812 None admin] PATH_INFO: `/v3/services/network` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:13.368751 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff5adeb-ab0c-4403-bfef-e076d80d7812 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:13.370344 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff5adeb-ab0c-4403-bfef-e076d80d7812 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:13.384218 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-1ff5adeb-ab0c-4403-bfef-e076d80d7812 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 17 19:11:13.384218 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:13.384218 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 17 19:11:13.384218 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 17 19:11:13.384218 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 17 19:11:13.384218 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 17 19:11:13.384218 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 17 19:11:13.384218 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 17 19:11:13.384218 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 17 19:11:13.384218 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:11:13.384218 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 17 19:11:13.384218 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:11:13.384218 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:13.384218 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:11:13.384218 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:11:13.384218 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:11:13.384218 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:11:13.385267 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:11:13.385267 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:11:13.385267 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:11:13.385267 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:11:13.385267 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:11:13.385267 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:11:13.385267 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 17 19:11:13.385267 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_service(service_id) May 17 19:11:13.385267 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 17 19:11:13.385267 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 17 19:11:13.385267 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:11:13.385267 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:11:13.385267 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:11:13.385267 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:11:13.385267 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:11:13.385267 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:11:13.386116 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:11:13.386116 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:11:13.386116 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:11:13.386116 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:11:13.386116 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:11:13.386116 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:11:13.386116 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:11:13.386116 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:11:13.386116 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:11:13.386116 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:11:13.386116 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 17 19:11:13.386116 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 17 19:11:13.386116 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 17 19:11:13.386116 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:11:13.392525 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 413/825] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 17 19:11:13 2022] GET /identity/v3/services/network => generated 88 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:11:13.395278 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff5adeb-ab0c-4403-bfef-e076d80d7812 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:13.425231 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11c070a-cb00-48bb-826e-586cd4416e53 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:13.501641 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff5adeb-ab0c-4403-bfef-e076d80d7812 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:13.506067 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1786baae-fe74-4ff4-890e-09aff189f410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:13.506735 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1786baae-fe74-4ff4-890e-09aff189f410 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:13.507018 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1786baae-fe74-4ff4-890e-09aff189f410 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:13.507289 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1786baae-fe74-4ff4-890e-09aff189f410 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:13.508041 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1786baae-fe74-4ff4-890e-09aff189f410 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:13.508343 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1786baae-fe74-4ff4-890e-09aff189f410 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:13.509716 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1786baae-fe74-4ff4-890e-09aff189f410 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:13.533894 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 413/826] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 17 19:11:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:13.540496 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1786baae-fe74-4ff4-890e-09aff189f410 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:13.542758 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f4aa59-2c7a-4214-b8b3-7fcee011e3a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:13.543696 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f4aa59-2c7a-4214-b8b3-7fcee011e3a5 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:13.544174 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f4aa59-2c7a-4214-b8b3-7fcee011e3a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:13.544666 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f4aa59-2c7a-4214-b8b3-7fcee011e3a5 None admin] PATH_INFO: `/v3/services` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:13.545564 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f4aa59-2c7a-4214-b8b3-7fcee011e3a5 None admin] RBAC: Adding query filter params (name=network) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:13.546060 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f4aa59-2c7a-4214-b8b3-7fcee011e3a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:13.547691 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f4aa59-2c7a-4214-b8b3-7fcee011e3a5 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:13.557939 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 414/827] 10.222.0.58 () {56 vars in 1112 bytes} [Tue May 17 19:11:13 2022] GET /identity/v3/services?name=network => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:13.559426 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f4aa59-2c7a-4214-b8b3-7fcee011e3a5 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:13.658821 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f4aa59-2c7a-4214-b8b3-7fcee011e3a5 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:13.696906 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c32ff2-99e5-4c2e-a8ec-f45e981397cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:13.700064 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c32ff2-99e5-4c2e-a8ec-f45e981397cf None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:13.700064 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c32ff2-99e5-4c2e-a8ec-f45e981397cf None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:13.700064 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c32ff2-99e5-4c2e-a8ec-f45e981397cf None admin] PATH_INFO: `/v3/services` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:13.700064 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c32ff2-99e5-4c2e-a8ec-f45e981397cf None admin] RBAC: Adding query filter params (type=network) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:13.700064 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c32ff2-99e5-4c2e-a8ec-f45e981397cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:13.701296 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c32ff2-99e5-4c2e-a8ec-f45e981397cf None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:13.711408 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 415/828] 10.222.0.58 () {56 vars in 1112 bytes} [Tue May 17 19:11:13 2022] GET /identity/v3/services?type=network => generated 350 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:11:13.713002 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c32ff2-99e5-4c2e-a8ec-f45e981397cf None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:13.752786 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1786baae-fe74-4ff4-890e-09aff189f410 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:13.789120 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2376d5f0-8785-4bf4-a198-43ebc7e89f6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:13.790213 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2376d5f0-8785-4bf4-a198-43ebc7e89f6b None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:13.790667 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2376d5f0-8785-4bf4-a198-43ebc7e89f6b None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:13.791133 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2376d5f0-8785-4bf4-a198-43ebc7e89f6b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:13.799103 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2376d5f0-8785-4bf4-a198-43ebc7e89f6b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:13.801410 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2376d5f0-8785-4bf4-a198-43ebc7e89f6b None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:13.817312 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-2376d5f0-8785-4bf4-a198-43ebc7e89f6b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 17 19:11:13.817312 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:13.817312 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:11:13.817312 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:11:13.817312 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:11:13.817312 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:11:13.817312 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:11:13.817312 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:11:13.817312 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:11:13.817312 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:11:13.817312 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:11:13.817312 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:11:13.817312 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 17 19:11:13.817312 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(project_id) May 17 19:11:13.817312 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 17 19:11:13.818336 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 17 19:11:13.818336 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:11:13.818336 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:11:13.818336 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:11:13.818336 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:11:13.818336 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:11:13.818336 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:11:13.818336 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:11:13.818336 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:11:13.818336 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:11:13.818336 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:11:13.818336 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:11:13.818336 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:11:13.818336 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:11:13.818336 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:11:13.818336 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:11:13.818336 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:11:13.819215 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 17 19:11:13.819215 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 17 19:11:13.819215 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 17 19:11:13.819215 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 17 19:11:13.819215 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 17 19:11:13.819215 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 17 19:11:13.819215 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 17 19:11:13.819215 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:11:13.826249 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 414/829] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 17 19:11:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 17 19:11:13.827711 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2376d5f0-8785-4bf4-a198-43ebc7e89f6b None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:13.882720 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c32ff2-99e5-4c2e-a8ec-f45e981397cf None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:13.916252 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8a057e-d4d8-4b3b-9faa-af9b11668e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:13.916831 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8a057e-d4d8-4b3b-9faa-af9b11668e4d None admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:13.917112 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8a057e-d4d8-4b3b-9faa-af9b11668e4d None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:13.917383 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8a057e-d4d8-4b3b-9faa-af9b11668e4d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:13.918127 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8a057e-d4d8-4b3b-9faa-af9b11668e4d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:13.919477 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8a057e-d4d8-4b3b-9faa-af9b11668e4d None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:13.935027 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2376d5f0-8785-4bf4-a198-43ebc7e89f6b None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:13.969507 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae696be-6f78-44d3-9e47-ce54a0370a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:13.970161 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae696be-6f78-44d3-9e47-ce54a0370a79 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:13.970441 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae696be-6f78-44d3-9e47-ce54a0370a79 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:13.970711 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae696be-6f78-44d3-9e47-ce54a0370a79 None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:13.972489 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae696be-6f78-44d3-9e47-ce54a0370a79 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:13.972798 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae696be-6f78-44d3-9e47-ce54a0370a79 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:13.974215 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae696be-6f78-44d3-9e47-ce54a0370a79 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:13.990488 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 415/830] 10.222.0.58 () {56 vars in 1148 bytes} [Tue May 17 19:11:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:11:13.990488 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8845bc1e-b866-4aa7-96e5-6ba8ca6fea4f None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:13.990488 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8845bc1e-b866-4aa7-96e5-6ba8ca6fea4f None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:13.990488 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8845bc1e-b866-4aa7-96e5-6ba8ca6fea4f None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:13.990488 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 416/831] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:11:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:11:13.991119 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8845bc1e-b866-4aa7-96e5-6ba8ca6fea4f None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:14.014171 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 416/832] 10.222.0.58 () {60 vars in 1144 bytes} [Tue May 17 19:11:13 2022] POST /identity/v3/endpoints => generated 327 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 17 19:11:14.015873 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6bd8d6-cf43-4f70-9c81-a62db146774d None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:14.016350 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6bd8d6-cf43-4f70-9c81-a62db146774d None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:14.016790 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6bd8d6-cf43-4f70-9c81-a62db146774d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:14.036997 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-2f6bd8d6-cf43-4f70-9c81-a62db146774d None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:11:14.038946 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-2f6bd8d6-cf43-4f70-9c81-a62db146774d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:11:14.106835 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8845bc1e-b866-4aa7-96e5-6ba8ca6fea4f None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:14.112703 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 417/833] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:11:14 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 17 19:11:14.147673 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2c82a0-b202-43c0-b535-ca4198abbb77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:14.148345 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2c82a0-b202-43c0-b535-ca4198abbb77 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:14.148690 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2c82a0-b202-43c0-b535-ca4198abbb77 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:14.148959 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2c82a0-b202-43c0-b535-ca4198abbb77 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:14.149860 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2c82a0-b202-43c0-b535-ca4198abbb77 None admin] RBAC: Adding query filter params (role.id=60af5088793e4379b88821a86552a344, scope.project.id=1a9d3a0631284c9fb113266036993e2d, user.id=65c1f878b0dc4d63a4f5dcb12fac1dc1) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:14.150166 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2c82a0-b202-43c0-b535-ca4198abbb77 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:14.151718 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2c82a0-b202-43c0-b535-ca4198abbb77 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:14.158156 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6bd8d6-cf43-4f70-9c81-a62db146774d None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:14.159039 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 417/834] 10.222.0.58 () {56 vars in 1374 bytes} [Tue May 17 19:11:13 2022] GET /identity/v3/role_assignments?user.id=65c1f878b0dc4d63a4f5dcb12fac1dc1&scope.project.id=1a9d3a0631284c9fb113266036993e2d&role.id=60af5088793e4379b88821a86552a344 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:11:14.304551 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6bd8d6-cf43-4f70-9c81-a62db146774d None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:14.342520 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c37f809-95c8-4ff9-a444-a3b40697bcc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:14.343322 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c37f809-95c8-4ff9-a444-a3b40697bcc6 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:14.343635 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c37f809-95c8-4ff9-a444-a3b40697bcc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:14.343909 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c37f809-95c8-4ff9-a444-a3b40697bcc6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:14.358777 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c37f809-95c8-4ff9-a444-a3b40697bcc6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 17 19:11:14.367228 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-0c37f809-95c8-4ff9-a444-a3b40697bcc6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 17 19:11:14.367228 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:14.367228 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:11:14.367228 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:11:14.367228 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:11:14.367228 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:11:14.367228 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:11:14.367228 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:11:14.367228 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:11:14.367228 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:11:14.367228 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:11:14.367228 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:11:14.367228 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 17 19:11:14.367228 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) May 17 19:11:14.367228 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 17 19:11:14.367228 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 17 19:11:14.368238 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 17 19:11:14.368238 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 17 19:11:14.368238 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 17 19:11:14.368238 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 17 19:11:14.368238 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:11:14.368238 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:11:14.368238 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:11:14.368238 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:11:14.368238 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:11:14.368238 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:11:14.368238 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:11:14.368238 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:11:14.368238 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:11:14.368238 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:11:14.368238 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:11:14.368238 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:11:14.369662 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:11:14.369662 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:11:14.369662 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:11:14.369662 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:11:14.369662 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 17 19:11:14.369662 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 17 19:11:14.369662 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 17 19:11:14.369662 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 17 19:11:14.369662 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 17 19:11:14.369662 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 17 19:11:14.369662 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 17 19:11:14.369662 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:11:14.369662 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 418/835] 10.222.0.58 () {56 vars in 1096 bytes} [Tue May 17 19:11:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 17 19:11:14.374414 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2c82a0-b202-43c0-b535-ca4198abbb77 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:14.491439 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2c82a0-b202-43c0-b535-ca4198abbb77 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:14.527644 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f1be63-cc71-46b3-ac64-6e208874b210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:14.528314 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f1be63-cc71-46b3-ac64-6e208874b210 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:14.528590 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f1be63-cc71-46b3-ac64-6e208874b210 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:14.528863 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f1be63-cc71-46b3-ac64-6e208874b210 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:14.529603 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f1be63-cc71-46b3-ac64-6e208874b210 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:14.529903 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f1be63-cc71-46b3-ac64-6e208874b210 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:14.531270 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f1be63-cc71-46b3-ac64-6e208874b210 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:14.540483 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 418/836] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 17 19:11:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:11:14.545748 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c37f809-95c8-4ff9-a444-a3b40697bcc6 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:14.654716 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c37f809-95c8-4ff9-a444-a3b40697bcc6 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:14.679097 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c9f4ca-0d1c-4ce4-8296-eb6c7f6678dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:14.679982 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c9f4ca-0d1c-4ce4-8296-eb6c7f6678dc None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:14.680262 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c9f4ca-0d1c-4ce4-8296-eb6c7f6678dc None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:14.680534 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c9f4ca-0d1c-4ce4-8296-eb6c7f6678dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:14.691818 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c9f4ca-0d1c-4ce4-8296-eb6c7f6678dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:14.693261 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c9f4ca-0d1c-4ce4-8296-eb6c7f6678dc None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:14.710489 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-93c9f4ca-0d1c-4ce4-8296-eb6c7f6678dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:11:14.710489 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:14.710489 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:11:14.710489 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:11:14.710489 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:11:14.710489 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:11:14.710489 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:11:14.710489 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:11:14.710489 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:11:14.710489 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:11:14.710489 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:11:14.710489 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:11:14.710489 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 17 19:11:14.710489 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 17 19:11:14.710489 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 17 19:11:14.711410 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 17 19:11:14.711410 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:11:14.711410 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:11:14.711410 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:11:14.711410 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:11:14.711410 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:11:14.711410 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:11:14.711410 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:11:14.711410 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:11:14.711410 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:11:14.711410 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:11:14.711410 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:11:14.711410 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:11:14.711410 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:11:14.711410 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:11:14.711410 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:11:14.711410 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:11:14.712337 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 17 19:11:14.712337 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 17 19:11:14.712337 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:11:14.712337 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:11:14.712337 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 419/837] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:11:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 17 19:11:14.718600 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f1be63-cc71-46b3-ac64-6e208874b210 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:14.819980 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f1be63-cc71-46b3-ac64-6e208874b210 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:14.853395 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2d6406-86a9-438d-9c33-404d05ecff37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:14.854060 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2d6406-86a9-438d-9c33-404d05ecff37 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:14.854336 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2d6406-86a9-438d-9c33-404d05ecff37 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:14.854607 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2d6406-86a9-438d-9c33-404d05ecff37 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:14.855329 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2d6406-86a9-438d-9c33-404d05ecff37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:14.855660 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2d6406-86a9-438d-9c33-404d05ecff37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:14.857041 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2d6406-86a9-438d-9c33-404d05ecff37 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:14.869996 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 419/838] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 17 19:11:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:11:14.875579 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c9f4ca-0d1c-4ce4-8296-eb6c7f6678dc None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:14.977087 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c9f4ca-0d1c-4ce4-8296-eb6c7f6678dc None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:15.010810 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7cebc0-140b-414a-8ffa-892e694a43c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:15.011627 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7cebc0-140b-414a-8ffa-892e694a43c7 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:15.011907 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7cebc0-140b-414a-8ffa-892e694a43c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:15.012180 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7cebc0-140b-414a-8ffa-892e694a43c7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:15.020382 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7cebc0-140b-414a-8ffa-892e694a43c7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:15.021881 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7cebc0-140b-414a-8ffa-892e694a43c7 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:15.035770 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-4f7cebc0-140b-414a-8ffa-892e694a43c7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 17 19:11:15.035770 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:15.035770 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:11:15.035770 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:11:15.035770 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:11:15.035770 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:11:15.035770 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:11:15.035770 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:11:15.035770 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:11:15.035770 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:11:15.035770 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:11:15.035770 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:11:15.035770 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 17 19:11:15.035770 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_user(user_id) May 17 19:11:15.035770 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 17 19:11:15.035770 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 17 19:11:15.036737 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:11:15.036737 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:11:15.036737 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 17 19:11:15.036737 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 17 19:11:15.036737 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 17 19:11:15.036737 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 17 19:11:15.036737 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:11:15.036737 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:11:15.036737 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:11:15.036737 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:11:15.036737 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:11:15.036737 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:11:15.036737 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:11:15.036737 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:11:15.036737 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:11:15.036737 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:11:15.036737 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:11:15.036737 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:11:15.037781 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:11:15.037781 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:11:15.037781 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 17 19:11:15.037781 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 17 19:11:15.037781 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 17 19:11:15.037781 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 17 19:11:15.037781 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 17 19:11:15.037781 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 17 19:11:15.037781 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 17 19:11:15.037781 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:11:15.037781 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 420/839] 10.222.0.58 () {56 vars in 1096 bytes} [Tue May 17 19:11:14 2022] GET /identity/v3/users/swift => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:11:15.044458 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2d6406-86a9-438d-9c33-404d05ecff37 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:15.149361 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2d6406-86a9-438d-9c33-404d05ecff37 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:15.186300 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0df9b5a-5f78-4eee-a53f-dcb448d17efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:15.186999 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0df9b5a-5f78-4eee-a53f-dcb448d17efc None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:15.187281 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0df9b5a-5f78-4eee-a53f-dcb448d17efc None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:15.187585 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0df9b5a-5f78-4eee-a53f-dcb448d17efc None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:15.188449 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0df9b5a-5f78-4eee-a53f-dcb448d17efc None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:15.188758 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0df9b5a-5f78-4eee-a53f-dcb448d17efc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:15.190307 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0df9b5a-5f78-4eee-a53f-dcb448d17efc None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:15.211289 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 420/840] 10.222.0.58 () {56 vars in 1135 bytes} [Tue May 17 19:11:15 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:11:15.216903 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7cebc0-140b-414a-8ffa-892e694a43c7 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:15.322126 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7cebc0-140b-414a-8ffa-892e694a43c7 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:15.356102 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05db5296-a0b1-4979-9447-81f49caafdba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:15.356744 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05db5296-a0b1-4979-9447-81f49caafdba None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:15.357024 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05db5296-a0b1-4979-9447-81f49caafdba None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:15.357299 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05db5296-a0b1-4979-9447-81f49caafdba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:15.374891 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201618ba-9551-4385-83ef-a1ba2ad3d48f None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:15.375857 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201618ba-9551-4385-83ef-a1ba2ad3d48f None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:15.376397 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201618ba-9551-4385-83ef-a1ba2ad3d48f None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:15.378760 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 421/841] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:11:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:11:15.383976 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c60431f-03d8-4f99-aeb3-839c0c104bd8 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:15.384479 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c60431f-03d8-4f99-aeb3-839c0c104bd8 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:15.384962 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c60431f-03d8-4f99-aeb3-839c0c104bd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:15.402202 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-2c60431f-03d8-4f99-aeb3-839c0c104bd8 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:11:15.403802 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-2c60431f-03d8-4f99-aeb3-839c0c104bd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:11:15.425965 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05db5296-a0b1-4979-9447-81f49caafdba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:15.427407 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05db5296-a0b1-4979-9447-81f49caafdba None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:15.484154 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 422/842] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:11:15 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 17 19:11:15.505372 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c60431f-03d8-4f99-aeb3-839c0c104bd8 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:15.540843 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 421/843] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 17 19:11:15 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 17 19:11:15.546038 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05db5296-a0b1-4979-9447-81f49caafdba None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:15.619306 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c60431f-03d8-4f99-aeb3-839c0c104bd8 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:15.652559 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05db5296-a0b1-4979-9447-81f49caafdba None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:15.659688 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e86b24d-84a6-49d9-9d44-5d6aaba42fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:15.660483 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e86b24d-84a6-49d9-9d44-5d6aaba42fd0 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:15.660756 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e86b24d-84a6-49d9-9d44-5d6aaba42fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:15.661021 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e86b24d-84a6-49d9-9d44-5d6aaba42fd0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:15.671345 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e86b24d-84a6-49d9-9d44-5d6aaba42fd0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 17 19:11:15.679873 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-9e86b24d-84a6-49d9-9d44-5d6aaba42fd0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 17 19:11:15.679873 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:15.679873 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:11:15.679873 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:11:15.679873 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:11:15.679873 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:11:15.679873 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:11:15.679873 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:11:15.679873 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:11:15.679873 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:11:15.679873 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:11:15.679873 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:11:15.679873 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 17 19:11:15.679873 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) May 17 19:11:15.679873 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 17 19:11:15.679873 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 17 19:11:15.681007 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 17 19:11:15.681007 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 17 19:11:15.681007 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 17 19:11:15.681007 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 17 19:11:15.681007 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:11:15.681007 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:11:15.681007 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:11:15.681007 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:11:15.681007 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:11:15.681007 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:11:15.681007 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:11:15.681007 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:11:15.681007 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:11:15.681007 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:11:15.681007 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:11:15.681007 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:11:15.682056 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:11:15.682056 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:11:15.682056 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:11:15.682056 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:11:15.682056 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 17 19:11:15.682056 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 17 19:11:15.682056 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 17 19:11:15.682056 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 17 19:11:15.682056 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 17 19:11:15.682056 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 17 19:11:15.682056 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 17 19:11:15.682056 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:11:15.684539 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 423/844] 10.222.0.58 () {56 vars in 1102 bytes} [Tue May 17 19:11:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 17 19:11:15.685898 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce970be-7d5e-4c7a-ab50-dc25a0bb7bf9 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:15.686180 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce970be-7d5e-4c7a-ab50-dc25a0bb7bf9 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:15.688231 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce970be-7d5e-4c7a-ab50-dc25a0bb7bf9 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:15.688231 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 424/845] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:11:15 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:11:15.692665 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce970be-7d5e-4c7a-ab50-dc25a0bb7bf9 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:15.693474 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f79e122-a5e6-4763-a995-9a6e37e0a931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:15.694268 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f79e122-a5e6-4763-a995-9a6e37e0a931 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:15.694546 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f79e122-a5e6-4763-a995-9a6e37e0a931 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:15.694815 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f79e122-a5e6-4763-a995-9a6e37e0a931 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:15.705114 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f79e122-a5e6-4763-a995-9a6e37e0a931 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:15.707158 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f79e122-a5e6-4763-a995-9a6e37e0a931 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:15.724555 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-0f79e122-a5e6-4763-a995-9a6e37e0a931 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:11:15.724555 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:15.724555 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:11:15.724555 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:11:15.724555 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:11:15.724555 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:11:15.724555 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:11:15.724555 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:11:15.724555 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:11:15.724555 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:11:15.724555 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:11:15.724555 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:11:15.724555 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 17 19:11:15.724555 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 17 19:11:15.724555 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 17 19:11:15.725461 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 17 19:11:15.725461 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:11:15.725461 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:11:15.725461 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:11:15.725461 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:11:15.725461 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:11:15.725461 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:11:15.725461 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:11:15.725461 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:11:15.725461 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:11:15.725461 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:11:15.725461 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:11:15.725461 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:11:15.725461 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:11:15.725461 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:11:15.725461 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:11:15.725461 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:11:15.726342 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 17 19:11:15.726342 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 17 19:11:15.726342 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:11:15.726342 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:11:15.726342 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 422/846] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:11:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 17 19:11:15.727098 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096234e4-c51c-4b9b-951a-8d762501c43d None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:15.727195 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096234e4-c51c-4b9b-951a-8d762501c43d None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:15.727447 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096234e4-c51c-4b9b-951a-8d762501c43d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:15.744734 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-096234e4-c51c-4b9b-951a-8d762501c43d None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:11:15.745843 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-096234e4-c51c-4b9b-951a-8d762501c43d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:11:15.799370 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce970be-7d5e-4c7a-ab50-dc25a0bb7bf9 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:15.810077 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 423/847] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:11:15 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 17 19:11:15.811075 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096234e4-c51c-4b9b-951a-8d762501c43d None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:15.845748 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956079c4-ef2e-4747-8b8d-14d76eace15c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:15.846690 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956079c4-ef2e-4747-8b8d-14d76eace15c None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:15.846968 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956079c4-ef2e-4747-8b8d-14d76eace15c None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:15.847240 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956079c4-ef2e-4747-8b8d-14d76eace15c None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:15.848013 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956079c4-ef2e-4747-8b8d-14d76eace15c None admin] RBAC: Adding query filter params (name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:15.848321 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956079c4-ef2e-4747-8b8d-14d76eace15c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:15.849732 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956079c4-ef2e-4747-8b8d-14d76eace15c None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:15.858043 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 425/848] 10.222.0.58 () {56 vars in 1103 bytes} [Tue May 17 19:11:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:15.858838 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956079c4-ef2e-4747-8b8d-14d76eace15c None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:15.916466 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096234e4-c51c-4b9b-951a-8d762501c43d None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:15.956671 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35627707-b541-49dc-97a3-04587068a79d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:15.957390 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35627707-b541-49dc-97a3-04587068a79d None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:15.957691 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35627707-b541-49dc-97a3-04587068a79d None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:15.957985 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35627707-b541-49dc-97a3-04587068a79d None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:15.958749 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35627707-b541-49dc-97a3-04587068a79d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:15.959052 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35627707-b541-49dc-97a3-04587068a79d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:15.960436 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35627707-b541-49dc-97a3-04587068a79d None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:15.967625 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956079c4-ef2e-4747-8b8d-14d76eace15c None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:15.972809 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 424/849] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 17 19:11:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:15.973622 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35627707-b541-49dc-97a3-04587068a79d None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:16.007836 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76549681-5fa7-4d92-b9fe-d80055bfb54f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:16.008036 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76549681-5fa7-4d92-b9fe-d80055bfb54f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:16.008124 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76549681-5fa7-4d92-b9fe-d80055bfb54f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:16.008395 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76549681-5fa7-4d92-b9fe-d80055bfb54f None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:16.009027 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76549681-5fa7-4d92-b9fe-d80055bfb54f None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:16.010382 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76549681-5fa7-4d92-b9fe-d80055bfb54f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:16.022385 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-76549681-5fa7-4d92-b9fe-d80055bfb54f None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 17 19:11:16.022385 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:16.022385 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 17 19:11:16.022385 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 17 19:11:16.022385 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 17 19:11:16.022385 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 17 19:11:16.022385 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 17 19:11:16.022385 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 17 19:11:16.022385 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 17 19:11:16.022385 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:11:16.022385 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 17 19:11:16.022385 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:11:16.022385 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:16.022385 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:11:16.022385 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:11:16.022385 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:11:16.022385 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:11:16.023464 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:11:16.023464 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:11:16.023464 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:11:16.023464 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:11:16.023464 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:11:16.023464 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:11:16.023464 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 17 19:11:16.023464 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(service_id) May 17 19:11:16.023464 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 17 19:11:16.023464 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 17 19:11:16.023464 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:11:16.023464 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:11:16.023464 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:11:16.023464 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:11:16.023464 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:11:16.023464 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:11:16.024432 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:11:16.024432 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:11:16.024432 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:11:16.024432 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:11:16.024432 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:11:16.024432 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:11:16.024432 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:11:16.024432 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:11:16.024432 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:11:16.024432 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:11:16.024432 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 17 19:11:16.024432 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 17 19:11:16.024432 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 17 19:11:16.024432 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:11:16.028625 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 426/850] 10.222.0.58 () {56 vars in 1114 bytes} [Tue May 17 19:11:15 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:11:16.029471 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76549681-5fa7-4d92-b9fe-d80055bfb54f None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:16.072690 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35627707-b541-49dc-97a3-04587068a79d None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:16.106131 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d08d191-64f4-4da1-89c9-55e751abdca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:16.106912 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d08d191-64f4-4da1-89c9-55e751abdca0 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:16.107193 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d08d191-64f4-4da1-89c9-55e751abdca0 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:16.107463 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d08d191-64f4-4da1-89c9-55e751abdca0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:16.117782 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d08d191-64f4-4da1-89c9-55e751abdca0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:16.119222 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d08d191-64f4-4da1-89c9-55e751abdca0 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:16.125445 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76549681-5fa7-4d92-b9fe-d80055bfb54f None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:16.136790 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-8d08d191-64f4-4da1-89c9-55e751abdca0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:11:16.136790 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:16.136790 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:11:16.136790 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:11:16.136790 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:11:16.136790 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:11:16.136790 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:11:16.136790 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:11:16.136790 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:11:16.136790 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:11:16.136790 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:11:16.136790 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:11:16.136790 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 17 19:11:16.136790 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 17 19:11:16.136790 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 17 19:11:16.137889 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 17 19:11:16.137889 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:11:16.137889 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:11:16.137889 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:11:16.137889 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:11:16.137889 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:11:16.137889 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:11:16.137889 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:11:16.137889 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:11:16.137889 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:11:16.137889 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:11:16.137889 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:11:16.137889 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:11:16.137889 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:11:16.137889 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:11:16.137889 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:11:16.137889 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:11:16.138999 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 17 19:11:16.138999 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 17 19:11:16.138999 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:11:16.138999 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:11:16.140568 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 425/851] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:11:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:11:16.141350 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d08d191-64f4-4da1-89c9-55e751abdca0 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:16.164232 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5267b11-3420-4197-8a39-51b3f6e0a71d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:16.165076 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5267b11-3420-4197-8a39-51b3f6e0a71d None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:16.165359 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5267b11-3420-4197-8a39-51b3f6e0a71d None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:16.165628 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5267b11-3420-4197-8a39-51b3f6e0a71d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:16.172428 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5267b11-3420-4197-8a39-51b3f6e0a71d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:16.173895 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5267b11-3420-4197-8a39-51b3f6e0a71d None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:16.187941 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-e5267b11-3420-4197-8a39-51b3f6e0a71d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 17 19:11:16.187941 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:16.187941 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:11:16.187941 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:11:16.187941 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:11:16.187941 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:11:16.187941 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:11:16.187941 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:11:16.187941 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:11:16.187941 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:11:16.187941 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:11:16.187941 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:11:16.187941 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 17 19:11:16.187941 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(project_id) May 17 19:11:16.187941 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 17 19:11:16.189055 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 17 19:11:16.189055 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:11:16.189055 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:11:16.189055 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:11:16.189055 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:11:16.189055 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:11:16.189055 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:11:16.189055 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:11:16.189055 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:11:16.189055 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:11:16.189055 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:11:16.189055 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:11:16.189055 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:11:16.189055 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:11:16.189055 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:11:16.189055 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:11:16.189055 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:11:16.189963 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 17 19:11:16.189963 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 17 19:11:16.189963 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 17 19:11:16.189963 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 17 19:11:16.189963 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 17 19:11:16.189963 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 17 19:11:16.189963 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 17 19:11:16.189963 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:11:16.189963 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 427/852] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 17 19:11:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:11:16.191227 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5267b11-3420-4197-8a39-51b3f6e0a71d None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:16.246896 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d08d191-64f4-4da1-89c9-55e751abdca0 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:16.279328 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00f206a-ea10-4758-88f3-5e957bf37d5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:16.279937 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00f206a-ea10-4758-88f3-5e957bf37d5d None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:16.280212 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00f206a-ea10-4758-88f3-5e957bf37d5d None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:16.280483 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00f206a-ea10-4758-88f3-5e957bf37d5d None admin] PATH_INFO: `/v3/services` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:16.281176 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00f206a-ea10-4758-88f3-5e957bf37d5d None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:16.281471 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00f206a-ea10-4758-88f3-5e957bf37d5d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:16.282814 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00f206a-ea10-4758-88f3-5e957bf37d5d None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:16.292347 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 426/853] 10.222.0.58 () {56 vars in 1114 bytes} [Tue May 17 19:11:16 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:16.292347 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00f206a-ea10-4758-88f3-5e957bf37d5d None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:16.299430 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5267b11-3420-4197-8a39-51b3f6e0a71d None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:16.332912 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae50b0c2-ae8a-43b3-a6b7-38be8a2efa51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:16.333575 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae50b0c2-ae8a-43b3-a6b7-38be8a2efa51 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:16.333858 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae50b0c2-ae8a-43b3-a6b7-38be8a2efa51 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:16.334132 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae50b0c2-ae8a-43b3-a6b7-38be8a2efa51 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:16.334844 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae50b0c2-ae8a-43b3-a6b7-38be8a2efa51 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:16.335145 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae50b0c2-ae8a-43b3-a6b7-38be8a2efa51 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:16.336551 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae50b0c2-ae8a-43b3-a6b7-38be8a2efa51 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:16.352491 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 428/854] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 17 19:11:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:16.352491 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae50b0c2-ae8a-43b3-a6b7-38be8a2efa51 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:16.390243 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00f206a-ea10-4758-88f3-5e957bf37d5d None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:16.421686 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe5f474-710c-494a-91a4-b499d800684d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:16.422330 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe5f474-710c-494a-91a4-b499d800684d None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:16.422605 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe5f474-710c-494a-91a4-b499d800684d None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:16.422869 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe5f474-710c-494a-91a4-b499d800684d None admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:16.423675 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe5f474-710c-494a-91a4-b499d800684d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:16.423978 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe5f474-710c-494a-91a4-b499d800684d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:16.425907 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe5f474-710c-494a-91a4-b499d800684d None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:16.437447 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 427/855] 10.222.0.58 () {56 vars in 1148 bytes} [Tue May 17 19:11:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:16.438234 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe5f474-710c-494a-91a4-b499d800684d None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:16.454351 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae50b0c2-ae8a-43b3-a6b7-38be8a2efa51 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:16.490382 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fb5d49-1aef-4882-b1ad-62f027783b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:16.491067 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fb5d49-1aef-4882-b1ad-62f027783b5b None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:16.491348 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fb5d49-1aef-4882-b1ad-62f027783b5b None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:16.491654 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fb5d49-1aef-4882-b1ad-62f027783b5b None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:16.492370 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fb5d49-1aef-4882-b1ad-62f027783b5b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:16.492677 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fb5d49-1aef-4882-b1ad-62f027783b5b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:16.494087 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fb5d49-1aef-4882-b1ad-62f027783b5b None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:16.503535 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 429/856] 10.222.0.58 () {56 vars in 1114 bytes} [Tue May 17 19:11:16 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:16.503535 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fb5d49-1aef-4882-b1ad-62f027783b5b None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:16.577638 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe5f474-710c-494a-91a4-b499d800684d None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:16.614914 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fb5d49-1aef-4882-b1ad-62f027783b5b None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:16.616974 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994e33a5-2079-40f7-a191-207ec61ce6a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:16.617866 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994e33a5-2079-40f7-a191-207ec61ce6a3 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:16.618247 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994e33a5-2079-40f7-a191-207ec61ce6a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:16.618615 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994e33a5-2079-40f7-a191-207ec61ce6a3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:16.626897 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994e33a5-2079-40f7-a191-207ec61ce6a3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:16.628644 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994e33a5-2079-40f7-a191-207ec61ce6a3 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:16.645342 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-994e33a5-2079-40f7-a191-207ec61ce6a3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 17 19:11:16.645342 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:16.645342 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:11:16.645342 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:11:16.645342 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:11:16.645342 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:11:16.645342 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:11:16.645342 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:11:16.645342 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:11:16.645342 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:11:16.645342 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:11:16.645342 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:11:16.645342 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 17 19:11:16.645342 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_user(user_id) May 17 19:11:16.645342 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 17 19:11:16.645342 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 17 19:11:16.651906 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:11:16.651906 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:11:16.651906 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 17 19:11:16.651906 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 17 19:11:16.651906 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 17 19:11:16.651906 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 17 19:11:16.651906 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:11:16.651906 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:11:16.651906 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:11:16.651906 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:11:16.651906 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:11:16.651906 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:11:16.651906 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:11:16.651906 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:11:16.651906 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:11:16.651906 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:11:16.651906 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:11:16.651906 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:11:16.652868 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:11:16.652868 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:11:16.652868 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 17 19:11:16.652868 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 17 19:11:16.652868 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 17 19:11:16.652868 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 17 19:11:16.652868 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 17 19:11:16.652868 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 17 19:11:16.652868 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 17 19:11:16.652868 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:11:16.652868 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 428/857] 10.222.0.58 () {56 vars in 1117 bytes} [Tue May 17 19:11:16 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:11:16.653484 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbf51ff-199c-4e56-816f-24bd90eaba3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:16.653484 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbf51ff-199c-4e56-816f-24bd90eaba3c None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:16.653639 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994e33a5-2079-40f7-a191-207ec61ce6a3 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:16.653707 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbf51ff-199c-4e56-816f-24bd90eaba3c None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:16.653707 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbf51ff-199c-4e56-816f-24bd90eaba3c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:16.659629 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbf51ff-199c-4e56-816f-24bd90eaba3c None admin] RBAC: Adding query filter params (role.id=60af5088793e4379b88821a86552a344, scope.project.id=1a9d3a0631284c9fb113266036993e2d, user.id=b0cb292002434d30bc81f370d120a2a2) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:16.659629 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbf51ff-199c-4e56-816f-24bd90eaba3c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:16.659629 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbf51ff-199c-4e56-816f-24bd90eaba3c None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:16.663950 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 430/858] 10.222.0.58 () {56 vars in 1374 bytes} [Tue May 17 19:11:16 2022] GET /identity/v3/role_assignments?user.id=b0cb292002434d30bc81f370d120a2a2&scope.project.id=1a9d3a0631284c9fb113266036993e2d&role.id=60af5088793e4379b88821a86552a344 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:16.667020 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b93b002-94df-4928-aeaf-8bd0fab4eff3 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:16.667544 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b93b002-94df-4928-aeaf-8bd0fab4eff3 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:16.668006 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b93b002-94df-4928-aeaf-8bd0fab4eff3 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:16.669862 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 431/859] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:11:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 17 19:11:16.677461 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd17f755-1c55-4aa7-a967-086eaf1bc81a None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:16.677808 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd17f755-1c55-4aa7-a967-086eaf1bc81a None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:16.678010 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd17f755-1c55-4aa7-a967-086eaf1bc81a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:16.696773 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-dd17f755-1c55-4aa7-a967-086eaf1bc81a None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:11:16.705885 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-dd17f755-1c55-4aa7-a967-086eaf1bc81a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:11:16.779246 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994e33a5-2079-40f7-a191-207ec61ce6a3 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:16.789601 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 432/860] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:11:16 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 17 19:11:16.817653 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd17f755-1c55-4aa7-a967-086eaf1bc81a None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:16.825618 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbc3d4a-e273-4f2f-9085-77d9a587a4b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:16.826692 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbc3d4a-e273-4f2f-9085-77d9a587a4b7 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:16.827179 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbc3d4a-e273-4f2f-9085-77d9a587a4b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:16.827790 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbc3d4a-e273-4f2f-9085-77d9a587a4b7 None admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:16.828870 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbc3d4a-e273-4f2f-9085-77d9a587a4b7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:16.829403 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbc3d4a-e273-4f2f-9085-77d9a587a4b7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:16.831669 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbc3d4a-e273-4f2f-9085-77d9a587a4b7 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:16.851180 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 429/861] 10.222.0.58 () {56 vars in 1149 bytes} [Tue May 17 19:11:16 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:11:16.861000 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbc3d4a-e273-4f2f-9085-77d9a587a4b7 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:16.926103 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd17f755-1c55-4aa7-a967-086eaf1bc81a None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:16.960904 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a13c58-c817-4d7b-a209-6562e1ba7e0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:16.961996 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a13c58-c817-4d7b-a209-6562e1ba7e0d None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:16.962291 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a13c58-c817-4d7b-a209-6562e1ba7e0d None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:16.962535 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a13c58-c817-4d7b-a209-6562e1ba7e0d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:16.963176 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a13c58-c817-4d7b-a209-6562e1ba7e0d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:16.964586 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a13c58-c817-4d7b-a209-6562e1ba7e0d None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:16.968268 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbc3d4a-e273-4f2f-9085-77d9a587a4b7 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:16.982321 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-76a13c58-c817-4d7b-a209-6562e1ba7e0d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 17 19:11:16.982321 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:16.982321 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 17 19:11:16.982321 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 17 19:11:16.982321 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 17 19:11:16.982321 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 17 19:11:16.982321 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 17 19:11:16.982321 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 17 19:11:16.982321 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 17 19:11:16.982321 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:11:16.982321 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 17 19:11:16.982321 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:11:16.982321 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:16.982321 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:11:16.982321 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:11:16.982321 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:11:16.984139 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:11:16.984139 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:11:16.984139 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:11:16.984139 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:11:16.984139 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:11:16.984139 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:11:16.984139 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:11:16.984139 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 17 19:11:16.984139 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(service_id) May 17 19:11:16.984139 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 17 19:11:16.984139 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 17 19:11:16.984139 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:11:16.984139 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:11:16.984139 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:11:16.984139 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:11:16.984139 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:11:16.985363 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:11:16.985363 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:11:16.985363 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:11:16.985363 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:11:16.985363 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:11:16.985363 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:11:16.985363 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:11:16.985363 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:11:16.985363 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:11:16.985363 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:11:16.985363 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:11:16.985363 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 17 19:11:16.985363 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 17 19:11:16.985363 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 17 19:11:16.985363 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:11:16.985363 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 433/862] 10.222.0.58 () {56 vars in 1132 bytes} [Tue May 17 19:11:16 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 17 19:11:16.993075 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a13c58-c817-4d7b-a209-6562e1ba7e0d None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:17.010239 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbae505-cd69-46d2-83f4-688a39ed51ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:17.010919 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbae505-cd69-46d2-83f4-688a39ed51ad None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:17.011197 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbae505-cd69-46d2-83f4-688a39ed51ad None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:17.011466 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbae505-cd69-46d2-83f4-688a39ed51ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:17.079733 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbae505-cd69-46d2-83f4-688a39ed51ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:17.081607 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbae505-cd69-46d2-83f4-688a39ed51ad None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:17.098421 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a13c58-c817-4d7b-a209-6562e1ba7e0d None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:17.132657 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd63c160-2b0e-49a8-baad-726267b86fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:17.133675 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd63c160-2b0e-49a8-baad-726267b86fe2 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:17.134226 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd63c160-2b0e-49a8-baad-726267b86fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:17.134737 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd63c160-2b0e-49a8-baad-726267b86fe2 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:17.135730 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd63c160-2b0e-49a8-baad-726267b86fe2 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:17.136240 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd63c160-2b0e-49a8-baad-726267b86fe2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:17.137788 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd63c160-2b0e-49a8-baad-726267b86fe2 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:17.147876 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 434/863] 10.222.0.58 () {56 vars in 1126 bytes} [Tue May 17 19:11:16 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:11:17.149093 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd63c160-2b0e-49a8-baad-726267b86fe2 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:17.184997 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 430/864] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 17 19:11:16 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 17 19:11:17.186284 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbae505-cd69-46d2-83f4-688a39ed51ad None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:17.250528 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd63c160-2b0e-49a8-baad-726267b86fe2 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:17.283795 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbae505-cd69-46d2-83f4-688a39ed51ad None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:17.285498 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f784df-6fed-465e-b035-88c8917c6b06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:17.286424 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f784df-6fed-465e-b035-88c8917c6b06 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:17.286939 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f784df-6fed-465e-b035-88c8917c6b06 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:17.287422 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f784df-6fed-465e-b035-88c8917c6b06 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:17.288363 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f784df-6fed-465e-b035-88c8917c6b06 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:17.288864 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f784df-6fed-465e-b035-88c8917c6b06 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:17.290435 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f784df-6fed-465e-b035-88c8917c6b06 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:17.303795 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 435/865] 10.222.0.58 () {56 vars in 1126 bytes} [Tue May 17 19:11:17 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:17.320707 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844aff2b-4d30-4d8e-a089-a9335d35a123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:17.321511 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844aff2b-4d30-4d8e-a089-a9335d35a123 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:17.321789 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844aff2b-4d30-4d8e-a089-a9335d35a123 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:17.322060 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844aff2b-4d30-4d8e-a089-a9335d35a123 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:17.334286 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844aff2b-4d30-4d8e-a089-a9335d35a123 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:17.335703 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844aff2b-4d30-4d8e-a089-a9335d35a123 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:17.355372 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-844aff2b-4d30-4d8e-a089-a9335d35a123 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:11:17.355372 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:17.355372 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:11:17.355372 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:11:17.355372 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:11:17.355372 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:11:17.355372 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:11:17.355372 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:11:17.355372 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:11:17.355372 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:11:17.355372 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:11:17.355372 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:11:17.355372 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 17 19:11:17.355372 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 17 19:11:17.355372 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 17 19:11:17.356378 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 17 19:11:17.356378 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:11:17.356378 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:11:17.356378 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:11:17.356378 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:11:17.356378 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:11:17.356378 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:11:17.356378 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:11:17.356378 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:11:17.356378 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:11:17.356378 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:11:17.356378 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:11:17.356378 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:11:17.356378 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:11:17.356378 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:11:17.356378 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:11:17.356378 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:11:17.357277 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 17 19:11:17.357277 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 17 19:11:17.357277 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:11:17.357277 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:11:17.361409 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 431/866] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:11:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:11:17.363441 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f784df-6fed-465e-b035-88c8917c6b06 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:17.474016 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f784df-6fed-465e-b035-88c8917c6b06 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:17.511205 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef2c66c-d159-48af-8784-a8ea0bcb5567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:17.511965 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef2c66c-d159-48af-8784-a8ea0bcb5567 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:17.512244 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef2c66c-d159-48af-8784-a8ea0bcb5567 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:17.512517 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef2c66c-d159-48af-8784-a8ea0bcb5567 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:17.513242 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef2c66c-d159-48af-8784-a8ea0bcb5567 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:17.513543 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef2c66c-d159-48af-8784-a8ea0bcb5567 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:17.514916 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef2c66c-d159-48af-8784-a8ea0bcb5567 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:17.531661 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 436/867] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 17 19:11:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:17.533077 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844aff2b-4d30-4d8e-a089-a9335d35a123 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:17.633930 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844aff2b-4d30-4d8e-a089-a9335d35a123 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:17.669457 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf598f4d-ec00-4a4a-a721-88af739b1e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:17.670261 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf598f4d-ec00-4a4a-a721-88af739b1e12 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:17.670543 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf598f4d-ec00-4a4a-a721-88af739b1e12 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:17.670882 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf598f4d-ec00-4a4a-a721-88af739b1e12 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:17.678909 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf598f4d-ec00-4a4a-a721-88af739b1e12 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:17.680384 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf598f4d-ec00-4a4a-a721-88af739b1e12 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:17.694257 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-cf598f4d-ec00-4a4a-a721-88af739b1e12 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 17 19:11:17.694257 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:17.694257 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:11:17.694257 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:11:17.694257 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:11:17.694257 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:11:17.694257 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:11:17.694257 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:11:17.694257 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:11:17.694257 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:11:17.694257 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:11:17.694257 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:11:17.694257 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 17 19:11:17.694257 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(project_id) May 17 19:11:17.694257 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 17 19:11:17.695157 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 17 19:11:17.695157 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:11:17.695157 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:11:17.695157 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:11:17.695157 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:11:17.695157 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:11:17.695157 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:11:17.695157 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:11:17.695157 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:11:17.695157 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:11:17.695157 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:11:17.695157 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:11:17.695157 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:11:17.695157 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:11:17.695157 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:11:17.695157 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:11:17.695157 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:11:17.696126 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 17 19:11:17.696126 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 17 19:11:17.696126 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 17 19:11:17.696126 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 17 19:11:17.696126 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 17 19:11:17.696126 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 17 19:11:17.696126 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 17 19:11:17.696126 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:11:17.699622 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 432/868] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 17 19:11:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:11:17.701613 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef2c66c-d159-48af-8784-a8ea0bcb5567 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:17.803063 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef2c66c-d159-48af-8784-a8ea0bcb5567 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:17.838258 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b12e1b-4060-478f-9cfe-a561fa45c555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:17.838919 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b12e1b-4060-478f-9cfe-a561fa45c555 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:17.839194 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b12e1b-4060-478f-9cfe-a561fa45c555 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:17.839464 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b12e1b-4060-478f-9cfe-a561fa45c555 None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:17.840271 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b12e1b-4060-478f-9cfe-a561fa45c555 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:17.840572 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b12e1b-4060-478f-9cfe-a561fa45c555 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:17.842006 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b12e1b-4060-478f-9cfe-a561fa45c555 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:17.859571 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 437/869] 10.222.0.58 () {56 vars in 1148 bytes} [Tue May 17 19:11:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:17.861051 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf598f4d-ec00-4a4a-a721-88af739b1e12 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:17.964440 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf598f4d-ec00-4a4a-a721-88af739b1e12 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:17.998497 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d44c0c-b839-4cbe-852e-d13b256572d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:17.999155 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d44c0c-b839-4cbe-852e-d13b256572d0 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:17.999434 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d44c0c-b839-4cbe-852e-d13b256572d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:17.999734 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d44c0c-b839-4cbe-852e-d13b256572d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:18.000563 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d44c0c-b839-4cbe-852e-d13b256572d0 None admin] RBAC: Adding query filter params (role.id=1754a95b778c4e20b53b20cb52ada92b, scope.project.id=1a9d3a0631284c9fb113266036993e2d, user.id=2b90b57c542440448579b3aadda8b241) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:18.000872 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d44c0c-b839-4cbe-852e-d13b256572d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:18.002293 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d44c0c-b839-4cbe-852e-d13b256572d0 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:18.015642 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 433/870] 10.222.0.58 () {56 vars in 1374 bytes} [Tue May 17 19:11:17 2022] GET /identity/v3/role_assignments?user.id=2b90b57c542440448579b3aadda8b241&scope.project.id=1a9d3a0631284c9fb113266036993e2d&role.id=1754a95b778c4e20b53b20cb52ada92b => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:18.782558 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8499ed23-eaae-47bf-9d96-03f7aad736e5 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:18.783624 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8499ed23-eaae-47bf-9d96-03f7aad736e5 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:18.784696 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8499ed23-eaae-47bf-9d96-03f7aad736e5 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:18.786476 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 438/871] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:11:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:11:18.791764 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d73420-5047-4bf2-adcc-34d2b963aa09 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:18.792263 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d73420-5047-4bf2-adcc-34d2b963aa09 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:18.792769 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d73420-5047-4bf2-adcc-34d2b963aa09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:18.810015 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-94d73420-5047-4bf2-adcc-34d2b963aa09 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:11:18.811199 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-94d73420-5047-4bf2-adcc-34d2b963aa09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:11:18.884496 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 434/872] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:11:18 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 17 19:11:18.907430 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8499ed23-eaae-47bf-9d96-03f7aad736e5 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:19.009005 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8499ed23-eaae-47bf-9d96-03f7aad736e5 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:19.036056 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcbb616-e334-4926-b93c-385999ce35b1 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:19.036640 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcbb616-e334-4926-b93c-385999ce35b1 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:19.037129 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcbb616-e334-4926-b93c-385999ce35b1 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:19.039019 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 435/873] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:11:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 17 19:11:19.044443 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56495e72-a6d2-4c09-a48a-429492100b57 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:19.044965 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56495e72-a6d2-4c09-a48a-429492100b57 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:19.045542 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56495e72-a6d2-4c09-a48a-429492100b57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:19.045903 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7857559a-554a-4e7a-86b7-580efa98dbeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:19.046562 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7857559a-554a-4e7a-86b7-580efa98dbeb None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:19.046836 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7857559a-554a-4e7a-86b7-580efa98dbeb None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:19.047102 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7857559a-554a-4e7a-86b7-580efa98dbeb None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:19.047834 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7857559a-554a-4e7a-86b7-580efa98dbeb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:19.049408 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7857559a-554a-4e7a-86b7-580efa98dbeb None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:19.064832 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-56495e72-a6d2-4c09-a48a-429492100b57 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:11:19.066823 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-56495e72-a6d2-4c09-a48a-429492100b57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:11:19.116761 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 439/874] 10.222.0.58 () {60 vars in 1141 bytes} [Tue May 17 19:11:18 2022] POST /identity/v3/services => generated 251 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:11:19.160734 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 436/875] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:11:19 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 17 19:11:19.183870 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7857559a-554a-4e7a-86b7-580efa98dbeb None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:19.296770 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7857559a-554a-4e7a-86b7-580efa98dbeb None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:19.333275 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b51bf34-fc10-450b-8e77-c7b6ca3434ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:19.334080 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b51bf34-fc10-450b-8e77-c7b6ca3434ff None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:19.334354 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b51bf34-fc10-450b-8e77-c7b6ca3434ff None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:19.334620 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b51bf34-fc10-450b-8e77-c7b6ca3434ff None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:19.335265 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b51bf34-fc10-450b-8e77-c7b6ca3434ff None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:19.336673 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b51bf34-fc10-450b-8e77-c7b6ca3434ff None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:19.350629 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-3b51bf34-fc10-450b-8e77-c7b6ca3434ff None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 17 19:11:19.350629 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:19.350629 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 17 19:11:19.350629 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 17 19:11:19.350629 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 17 19:11:19.350629 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 17 19:11:19.350629 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 17 19:11:19.350629 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 17 19:11:19.350629 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 17 19:11:19.350629 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:11:19.350629 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 17 19:11:19.350629 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:11:19.350629 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:19.350629 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:11:19.350629 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:11:19.350629 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:11:19.350629 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:11:19.351805 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:11:19.351805 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:11:19.351805 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:11:19.351805 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:11:19.351805 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:11:19.351805 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:11:19.351805 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 17 19:11:19.351805 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(service_id) May 17 19:11:19.351805 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 17 19:11:19.351805 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 17 19:11:19.351805 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:11:19.351805 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:11:19.351805 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:11:19.351805 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:11:19.351805 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:11:19.351805 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:11:19.352648 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:11:19.352648 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:11:19.352648 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:11:19.352648 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:11:19.352648 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:11:19.352648 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:11:19.352648 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:11:19.352648 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:11:19.352648 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:11:19.352648 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:11:19.352648 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 17 19:11:19.352648 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 17 19:11:19.352648 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 17 19:11:19.352648 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:11:19.352648 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 440/876] 10.222.0.58 () {56 vars in 1126 bytes} [Tue May 17 19:11:19 2022] GET /identity/v3/services/object-store => generated 93 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:11:19.360548 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56495e72-a6d2-4c09-a48a-429492100b57 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:19.464543 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56495e72-a6d2-4c09-a48a-429492100b57 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:19.497444 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93c420c-9f28-4e77-94a0-7ffdbb2e3c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:19.498100 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93c420c-9f28-4e77-94a0-7ffdbb2e3c9c None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:19.498373 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93c420c-9f28-4e77-94a0-7ffdbb2e3c9c None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:19.498640 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93c420c-9f28-4e77-94a0-7ffdbb2e3c9c None admin] PATH_INFO: `/v3/services` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:19.499333 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93c420c-9f28-4e77-94a0-7ffdbb2e3c9c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:19.499660 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93c420c-9f28-4e77-94a0-7ffdbb2e3c9c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:19.501015 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93c420c-9f28-4e77-94a0-7ffdbb2e3c9c None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:19.506470 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 437/877] 10.222.0.58 () {56 vars in 1122 bytes} [Tue May 17 19:11:19 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:19.512061 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b51bf34-fc10-450b-8e77-c7b6ca3434ff None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:19.549321 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b780b015-982e-4396-8d49-56d84bbebe9f None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:19.550499 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b780b015-982e-4396-8d49-56d84bbebe9f None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:19.550985 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b780b015-982e-4396-8d49-56d84bbebe9f None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:19.553240 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 438/878] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:11:19 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 17 19:11:19.559368 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccb7817-21f2-4d03-8384-954753cf10eb None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:19.559670 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccb7817-21f2-4d03-8384-954753cf10eb None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:19.559942 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccb7817-21f2-4d03-8384-954753cf10eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:19.578557 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-8ccb7817-21f2-4d03-8384-954753cf10eb None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:11:19.579793 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-8ccb7817-21f2-4d03-8384-954753cf10eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:11:19.632950 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b51bf34-fc10-450b-8e77-c7b6ca3434ff None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:19.651557 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 439/879] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:11:19 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 17 19:11:19.669669 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccb7817-21f2-4d03-8384-954753cf10eb None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:19.681342 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cbedd3-81dd-4dfe-a252-ca79f1ddb7b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:19.682290 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cbedd3-81dd-4dfe-a252-ca79f1ddb7b7 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:19.682714 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cbedd3-81dd-4dfe-a252-ca79f1ddb7b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:19.683133 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cbedd3-81dd-4dfe-a252-ca79f1ddb7b7 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:19.684084 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cbedd3-81dd-4dfe-a252-ca79f1ddb7b7 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:19.684547 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cbedd3-81dd-4dfe-a252-ca79f1ddb7b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:19.686243 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cbedd3-81dd-4dfe-a252-ca79f1ddb7b7 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:19.692183 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 441/880] 10.222.0.58 () {56 vars in 1122 bytes} [Tue May 17 19:11:19 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:19.780284 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccb7817-21f2-4d03-8384-954753cf10eb None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:19.817990 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5e0541-fad4-4f58-b991-f0c839297105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:19.818587 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5e0541-fad4-4f58-b991-f0c839297105 None admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:19.818869 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5e0541-fad4-4f58-b991-f0c839297105 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:19.819145 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5e0541-fad4-4f58-b991-f0c839297105 None admin] PATH_INFO: `/v3/services` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:19.819920 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5e0541-fad4-4f58-b991-f0c839297105 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:19.821295 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5e0541-fad4-4f58-b991-f0c839297105 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:19.920932 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 440/881] 10.222.0.58 () {60 vars in 1141 bytes} [Tue May 17 19:11:19 2022] POST /identity/v3/services => generated 269 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:11:20.322453 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1712be5a-88a8-4ea3-8261-f230dbf04775 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:20.322976 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1712be5a-88a8-4ea3-8261-f230dbf04775 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:20.322976 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1712be5a-88a8-4ea3-8261-f230dbf04775 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:20.324613 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 442/882] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:11:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:11:20.329729 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef686f6c-8fef-46fd-ae1c-bb7e7f5ebbe8 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:20.330015 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef686f6c-8fef-46fd-ae1c-bb7e7f5ebbe8 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:20.330290 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef686f6c-8fef-46fd-ae1c-bb7e7f5ebbe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:20.349592 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-ef686f6c-8fef-46fd-ae1c-bb7e7f5ebbe8 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:11:20.350780 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-ef686f6c-8fef-46fd-ae1c-bb7e7f5ebbe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:11:20.423835 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 441/883] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:11:20 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 17 19:11:20.448191 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1712be5a-88a8-4ea3-8261-f230dbf04775 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:20.550919 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1712be5a-88a8-4ea3-8261-f230dbf04775 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:20.585301 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c554ea4-aef4-4378-bace-8eab164437b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:20.586422 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c554ea4-aef4-4378-bace-8eab164437b1 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:20.586921 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c554ea4-aef4-4378-bace-8eab164437b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:20.587392 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c554ea4-aef4-4378-bace-8eab164437b1 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:20.599112 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c554ea4-aef4-4378-bace-8eab164437b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 17 19:11:20.607427 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-9c554ea4-aef4-4378-bace-8eab164437b1 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 17 19:11:20.607427 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:20.607427 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:11:20.607427 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:11:20.607427 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:11:20.607427 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:11:20.607427 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:11:20.607427 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:11:20.607427 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:11:20.607427 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:11:20.607427 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:11:20.607427 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:11:20.607427 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 17 19:11:20.607427 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) May 17 19:11:20.607427 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 17 19:11:20.607427 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 17 19:11:20.608392 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 17 19:11:20.608392 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 17 19:11:20.608392 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 17 19:11:20.608392 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 17 19:11:20.608392 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:11:20.608392 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:11:20.608392 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:11:20.608392 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:11:20.608392 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:11:20.608392 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:11:20.608392 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:11:20.608392 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:11:20.608392 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:11:20.608392 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:11:20.608392 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:11:20.608392 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:11:20.609228 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:11:20.609228 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:11:20.609228 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:11:20.609228 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:11:20.609228 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 17 19:11:20.609228 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 17 19:11:20.609228 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 17 19:11:20.609228 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 17 19:11:20.609228 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 17 19:11:20.609228 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 17 19:11:20.609228 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 17 19:11:20.609228 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:11:20.611611 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 443/884] 10.222.0.58 () {56 vars in 1120 bytes} [Tue May 17 19:11:20 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:11:20.620744 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef686f6c-8fef-46fd-ae1c-bb7e7f5ebbe8 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:20.714167 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef686f6c-8fef-46fd-ae1c-bb7e7f5ebbe8 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:20.749263 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d27381-b799-4df4-8b7f-e6812ee85d0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:20.754194 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d27381-b799-4df4-8b7f-e6812ee85d0c None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:20.754194 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d27381-b799-4df4-8b7f-e6812ee85d0c None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:20.754194 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d27381-b799-4df4-8b7f-e6812ee85d0c None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:20.754194 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d27381-b799-4df4-8b7f-e6812ee85d0c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:20.754194 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d27381-b799-4df4-8b7f-e6812ee85d0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:20.754194 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d27381-b799-4df4-8b7f-e6812ee85d0c None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:20.762012 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 442/885] 10.222.0.58 () {56 vars in 1115 bytes} [Tue May 17 19:11:20 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:20.767469 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c554ea4-aef4-4378-bace-8eab164437b1 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:20.874653 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c554ea4-aef4-4378-bace-8eab164437b1 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:20.909036 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7df071-bf35-415b-ae83-fd0c728ef5bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:20.910139 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7df071-bf35-415b-ae83-fd0c728ef5bb None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:20.910597 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7df071-bf35-415b-ae83-fd0c728ef5bb None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:20.911047 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7df071-bf35-415b-ae83-fd0c728ef5bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:20.921576 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7df071-bf35-415b-ae83-fd0c728ef5bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:20.923233 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7df071-bf35-415b-ae83-fd0c728ef5bb None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:20.943870 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-5f7df071-bf35-415b-ae83-fd0c728ef5bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:11:20.943870 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:20.943870 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:11:20.943870 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:11:20.943870 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:11:20.943870 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:11:20.943870 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:11:20.943870 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:11:20.943870 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:11:20.943870 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:11:20.943870 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:11:20.943870 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:11:20.943870 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 17 19:11:20.943870 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 17 19:11:20.943870 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 17 19:11:20.944819 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 17 19:11:20.944819 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:11:20.944819 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:11:20.944819 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:11:20.944819 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:11:20.944819 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:11:20.944819 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:11:20.944819 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:11:20.944819 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:11:20.944819 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:11:20.944819 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:11:20.944819 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:11:20.944819 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:11:20.944819 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:11:20.944819 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:11:20.944819 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:11:20.944819 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:11:20.945798 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 17 19:11:20.945798 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 17 19:11:20.945798 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:11:20.945798 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:11:20.945798 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 444/886] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:11:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:11:20.948407 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d27381-b799-4df4-8b7f-e6812ee85d0c None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:21.050664 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d27381-b799-4df4-8b7f-e6812ee85d0c None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:21.087514 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b886ba4a-5a6f-4b6c-af85-9dca5bfe477c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:21.088177 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b886ba4a-5a6f-4b6c-af85-9dca5bfe477c None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:21.088460 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b886ba4a-5a6f-4b6c-af85-9dca5bfe477c None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:21.088730 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b886ba4a-5a6f-4b6c-af85-9dca5bfe477c None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:21.089434 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b886ba4a-5a6f-4b6c-af85-9dca5bfe477c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:21.089729 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b886ba4a-5a6f-4b6c-af85-9dca5bfe477c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:21.091079 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b886ba4a-5a6f-4b6c-af85-9dca5bfe477c None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:21.103472 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 443/887] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 17 19:11:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:21.108743 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7df071-bf35-415b-ae83-fd0c728ef5bb None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:21.213969 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7df071-bf35-415b-ae83-fd0c728ef5bb None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:21.248730 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde7f814-1737-4f85-9987-4805fc3f956c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:21.249908 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde7f814-1737-4f85-9987-4805fc3f956c None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:21.250401 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde7f814-1737-4f85-9987-4805fc3f956c None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:21.250897 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde7f814-1737-4f85-9987-4805fc3f956c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:21.259413 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde7f814-1737-4f85-9987-4805fc3f956c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:21.261107 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde7f814-1737-4f85-9987-4805fc3f956c None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:21.276000 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-bde7f814-1737-4f85-9987-4805fc3f956c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 17 19:11:21.276000 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:21.276000 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:11:21.276000 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:11:21.276000 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:11:21.276000 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:11:21.276000 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:11:21.276000 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:11:21.276000 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:11:21.276000 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:11:21.276000 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:11:21.276000 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:11:21.276000 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 17 19:11:21.276000 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_user(user_id) May 17 19:11:21.276000 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 17 19:11:21.276000 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 17 19:11:21.276982 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:11:21.276982 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:11:21.276982 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 17 19:11:21.276982 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 17 19:11:21.276982 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 17 19:11:21.276982 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 17 19:11:21.276982 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:11:21.276982 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:11:21.276982 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:11:21.276982 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:11:21.276982 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:11:21.276982 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:11:21.276982 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:11:21.276982 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:11:21.276982 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:11:21.276982 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:11:21.276982 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:11:21.276982 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:11:21.277932 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:11:21.277932 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:11:21.277932 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 17 19:11:21.277932 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 17 19:11:21.277932 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 17 19:11:21.277932 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 17 19:11:21.277932 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 17 19:11:21.277932 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 17 19:11:21.277932 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 17 19:11:21.277932 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:11:21.280522 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 445/888] 10.222.0.58 () {56 vars in 1117 bytes} [Tue May 17 19:11:21 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:11:21.286878 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b886ba4a-5a6f-4b6c-af85-9dca5bfe477c None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:21.392672 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b886ba4a-5a6f-4b6c-af85-9dca5bfe477c None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:21.427625 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dae9b56-24f5-49fb-b530-046a7bb4d0cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:21.428290 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dae9b56-24f5-49fb-b530-046a7bb4d0cc None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:21.428562 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dae9b56-24f5-49fb-b530-046a7bb4d0cc None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:21.428829 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dae9b56-24f5-49fb-b530-046a7bb4d0cc None admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:21.429656 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dae9b56-24f5-49fb-b530-046a7bb4d0cc None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:21.429955 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dae9b56-24f5-49fb-b530-046a7bb4d0cc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:21.431365 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dae9b56-24f5-49fb-b530-046a7bb4d0cc None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:21.450540 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 444/889] 10.222.0.58 () {56 vars in 1149 bytes} [Tue May 17 19:11:21 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:21.456252 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde7f814-1737-4f85-9987-4805fc3f956c None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:21.500887 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b5b85a-73c4-4302-9c6a-47877cd0feac None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:21.501073 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b5b85a-73c4-4302-9c6a-47877cd0feac None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:21.501347 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b5b85a-73c4-4302-9c6a-47877cd0feac None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:21.503478 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 445/890] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:11:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 17 19:11:21.508414 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eba092-5b47-4c8b-b9f5-0af048cde5e0 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:21.508696 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eba092-5b47-4c8b-b9f5-0af048cde5e0 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:21.508977 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eba092-5b47-4c8b-b9f5-0af048cde5e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:21.525106 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-d5eba092-5b47-4c8b-b9f5-0af048cde5e0 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:11:21.526275 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-d5eba092-5b47-4c8b-b9f5-0af048cde5e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:11:21.571218 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde7f814-1737-4f85-9987-4805fc3f956c None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:21.607651 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 446/891] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:11:21 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 17 19:11:21.612237 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07246e3-39f9-4b95-a659-8e2a0c0f0153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:21.612886 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07246e3-39f9-4b95-a659-8e2a0c0f0153 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:21.613167 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07246e3-39f9-4b95-a659-8e2a0c0f0153 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:21.613439 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07246e3-39f9-4b95-a659-8e2a0c0f0153 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:21.634406 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eba092-5b47-4c8b-b9f5-0af048cde5e0 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:21.684282 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07246e3-39f9-4b95-a659-8e2a0c0f0153 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:21.686114 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07246e3-39f9-4b95-a659-8e2a0c0f0153 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:21.734156 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eba092-5b47-4c8b-b9f5-0af048cde5e0 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:21.771528 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77605694-291a-417e-95f9-99fce47a2426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:21.771528 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77605694-291a-417e-95f9-99fce47a2426 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:21.771528 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77605694-291a-417e-95f9-99fce47a2426 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:21.771528 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77605694-291a-417e-95f9-99fce47a2426 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:21.771919 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77605694-291a-417e-95f9-99fce47a2426 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:21.773272 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77605694-291a-417e-95f9-99fce47a2426 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:21.786387 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-77605694-291a-417e-95f9-99fce47a2426 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 17 19:11:21.786387 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:21.786387 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 17 19:11:21.786387 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 17 19:11:21.786387 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 17 19:11:21.786387 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 17 19:11:21.786387 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 17 19:11:21.786387 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 17 19:11:21.786387 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 17 19:11:21.786387 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:11:21.786387 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 17 19:11:21.786387 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:11:21.786387 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:21.786387 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:11:21.786387 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:11:21.786387 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:11:21.786387 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:11:21.787510 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:11:21.787510 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:11:21.787510 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:11:21.787510 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:11:21.787510 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:11:21.787510 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:11:21.787510 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 17 19:11:21.787510 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_service(service_id) May 17 19:11:21.787510 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 17 19:11:21.787510 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 17 19:11:21.787510 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:11:21.787510 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:11:21.787510 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:11:21.787510 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:11:21.787510 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:11:21.787510 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:11:21.788372 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:11:21.788372 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:11:21.788372 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:11:21.788372 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:11:21.788372 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:11:21.788372 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:11:21.788372 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:11:21.788372 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:11:21.788372 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:11:21.788372 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:11:21.788372 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 17 19:11:21.788372 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 17 19:11:21.788372 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 17 19:11:21.788372 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:11:21.792956 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 447/892] 10.222.0.58 () {56 vars in 1114 bytes} [Tue May 17 19:11:21 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 17 19:11:21.793981 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77605694-291a-417e-95f9-99fce47a2426 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:21.799028 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 446/893] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 17 19:11:21 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 17 19:11:21.801454 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07246e3-39f9-4b95-a659-8e2a0c0f0153 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:21.902589 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77605694-291a-417e-95f9-99fce47a2426 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:21.906450 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07246e3-39f9-4b95-a659-8e2a0c0f0153 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:21.936401 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26ce84c-d639-4d8a-a1c4-b1cdbc7eec98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:21.937051 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26ce84c-d639-4d8a-a1c4-b1cdbc7eec98 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:21.937332 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26ce84c-d639-4d8a-a1c4-b1cdbc7eec98 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:21.937608 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26ce84c-d639-4d8a-a1c4-b1cdbc7eec98 None admin] PATH_INFO: `/v3/services` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:21.938315 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26ce84c-d639-4d8a-a1c4-b1cdbc7eec98 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:21.938619 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26ce84c-d639-4d8a-a1c4-b1cdbc7eec98 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:21.940086 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26ce84c-d639-4d8a-a1c4-b1cdbc7eec98 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:21.945662 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d16289-24fc-471b-b16a-80ba2878885b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:21.945662 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d16289-24fc-471b-b16a-80ba2878885b None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:21.945662 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d16289-24fc-471b-b16a-80ba2878885b None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:21.945662 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d16289-24fc-471b-b16a-80ba2878885b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:21.951558 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 448/894] 10.222.0.58 () {56 vars in 1114 bytes} [Tue May 17 19:11:21 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:21.953542 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26ce84c-d639-4d8a-a1c4-b1cdbc7eec98 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:21.962806 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d16289-24fc-471b-b16a-80ba2878885b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:21.964306 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d16289-24fc-471b-b16a-80ba2878885b None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:21.982955 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-a4d16289-24fc-471b-b16a-80ba2878885b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:11:21.982955 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:21.982955 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:11:21.982955 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:11:21.982955 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:11:21.982955 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:11:21.982955 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:11:21.982955 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:11:21.982955 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:11:21.982955 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:11:21.982955 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:11:21.982955 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:11:21.982955 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 17 19:11:21.982955 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 17 19:11:21.982955 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 17 19:11:21.983969 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 17 19:11:21.983969 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:11:21.983969 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:11:21.983969 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:11:21.983969 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:11:21.983969 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:11:21.983969 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:11:21.983969 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:11:21.983969 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:11:21.983969 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:11:21.983969 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:11:21.983969 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:11:21.983969 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:11:21.983969 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:11:21.983969 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:11:21.983969 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:11:21.983969 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:11:21.984876 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 17 19:11:21.984876 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 17 19:11:21.984876 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:11:21.984876 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:11:21.984876 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 447/895] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:11:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 17 19:11:21.985698 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5785fd-c280-4deb-9947-976f0b7d6028 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:21.985978 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5785fd-c280-4deb-9947-976f0b7d6028 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:21.986251 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5785fd-c280-4deb-9947-976f0b7d6028 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:21.987717 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 448/896] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:11:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:11:21.990992 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5785fd-c280-4deb-9947-976f0b7d6028 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:22.062264 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26ce84c-d639-4d8a-a1c4-b1cdbc7eec98 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:22.094326 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5785fd-c280-4deb-9947-976f0b7d6028 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:22.096174 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a6a2c4-313c-418a-9e82-2a56694363f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:22.097529 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a6a2c4-313c-418a-9e82-2a56694363f6 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:22.097803 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a6a2c4-313c-418a-9e82-2a56694363f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:22.098074 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a6a2c4-313c-418a-9e82-2a56694363f6 None admin] PATH_INFO: `/v3/services` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:22.098776 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a6a2c4-313c-418a-9e82-2a56694363f6 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:22.099099 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a6a2c4-313c-418a-9e82-2a56694363f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:22.100488 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a6a2c4-313c-418a-9e82-2a56694363f6 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:22.105332 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 449/897] 10.222.0.58 () {56 vars in 1114 bytes} [Tue May 17 19:11:21 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:11:22.106536 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f815b6-7f92-4b9c-9519-b3402bcf7a7b None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:22.106819 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f815b6-7f92-4b9c-9519-b3402bcf7a7b None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:22.107091 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f815b6-7f92-4b9c-9519-b3402bcf7a7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:22.122895 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-48f815b6-7f92-4b9c-9519-b3402bcf7a7b None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:11:22.124112 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-48f815b6-7f92-4b9c-9519-b3402bcf7a7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:11:22.133041 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1004cf-9090-4781-b319-c6f371c11efa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:22.133697 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1004cf-9090-4781-b319-c6f371c11efa None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:22.133978 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1004cf-9090-4781-b319-c6f371c11efa None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:22.134249 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1004cf-9090-4781-b319-c6f371c11efa None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:22.134952 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1004cf-9090-4781-b319-c6f371c11efa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:22.135257 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1004cf-9090-4781-b319-c6f371c11efa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:22.136657 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1004cf-9090-4781-b319-c6f371c11efa None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:22.147900 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 449/898] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 17 19:11:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:22.153005 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1004cf-9090-4781-b319-c6f371c11efa None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:22.191638 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 450/899] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:11:22 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 17 19:11:22.191638 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f815b6-7f92-4b9c-9519-b3402bcf7a7b None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:22.252035 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1004cf-9090-4781-b319-c6f371c11efa None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:22.284222 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f98e0b-a434-4859-9401-c5455f005e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:22.284868 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f98e0b-a434-4859-9401-c5455f005e9e None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:22.285146 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f98e0b-a434-4859-9401-c5455f005e9e None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:22.285412 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f98e0b-a434-4859-9401-c5455f005e9e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:22.286185 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f98e0b-a434-4859-9401-c5455f005e9e None admin] RBAC: Adding query filter params (interface=public, service_id=a69963cf07ea4629bc9a2134cd8abe61, region_id=RegionOne) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:22.286488 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f98e0b-a434-4859-9401-c5455f005e9e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:22.287892 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f98e0b-a434-4859-9401-c5455f005e9e None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:22.290360 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f815b6-7f92-4b9c-9519-b3402bcf7a7b None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:22.292668 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 450/900] 10.222.0.58 () {56 vars in 1251 bytes} [Tue May 17 19:11:22 2022] GET /identity/v3/endpoints?service_id=a69963cf07ea4629bc9a2134cd8abe61&interface=public®ion_id=RegionOne => generated 194 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:22.293403 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f98e0b-a434-4859-9401-c5455f005e9e None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:22.323281 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bd8872-f762-41bb-9c75-d3cae3352229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:22.324024 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bd8872-f762-41bb-9c75-d3cae3352229 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:22.324304 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bd8872-f762-41bb-9c75-d3cae3352229 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:22.324576 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bd8872-f762-41bb-9c75-d3cae3352229 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:22.331529 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bd8872-f762-41bb-9c75-d3cae3352229 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:22.332957 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bd8872-f762-41bb-9c75-d3cae3352229 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:22.347003 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-01bd8872-f762-41bb-9c75-d3cae3352229 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 17 19:11:22.347003 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:22.347003 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:11:22.347003 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:11:22.347003 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:11:22.347003 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:11:22.347003 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:11:22.347003 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:11:22.347003 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:11:22.347003 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:11:22.347003 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:11:22.347003 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:11:22.347003 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 17 19:11:22.347003 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(project_id) May 17 19:11:22.347003 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 17 19:11:22.347933 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 17 19:11:22.347933 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:11:22.347933 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:11:22.347933 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:11:22.347933 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:11:22.347933 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:11:22.347933 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:11:22.347933 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:11:22.347933 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:11:22.347933 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:11:22.347933 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:11:22.347933 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:11:22.347933 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:11:22.347933 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:11:22.347933 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:11:22.347933 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:11:22.347933 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:11:22.348864 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 17 19:11:22.348864 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 17 19:11:22.348864 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 17 19:11:22.348864 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 17 19:11:22.348864 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 17 19:11:22.348864 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 17 19:11:22.348864 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 17 19:11:22.348864 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:11:22.348864 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 451/901] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 17 19:11:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:11:22.349666 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32de7489-6114-418e-ab03-4e9a1a4ad82a None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:22.349956 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32de7489-6114-418e-ab03-4e9a1a4ad82a None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:22.350206 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32de7489-6114-418e-ab03-4e9a1a4ad82a None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:22.351606 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 452/902] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:11:22 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:11:22.352147 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32de7489-6114-418e-ab03-4e9a1a4ad82a None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:22.394869 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f98e0b-a434-4859-9401-c5455f005e9e None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:22.426441 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd62d844-65d1-47f6-a92e-730f94dfa9e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:22.427097 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd62d844-65d1-47f6-a92e-730f94dfa9e4 None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:22.427378 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd62d844-65d1-47f6-a92e-730f94dfa9e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:22.427676 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd62d844-65d1-47f6-a92e-730f94dfa9e4 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:22.428382 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd62d844-65d1-47f6-a92e-730f94dfa9e4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:22.429744 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd62d844-65d1-47f6-a92e-730f94dfa9e4 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:22.447403 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32de7489-6114-418e-ab03-4e9a1a4ad82a None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:22.479928 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753ecf6b-e686-4a33-a426-80a3d24da350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:22.480571 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753ecf6b-e686-4a33-a426-80a3d24da350 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:22.480852 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753ecf6b-e686-4a33-a426-80a3d24da350 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:22.481122 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753ecf6b-e686-4a33-a426-80a3d24da350 None admin] PATH_INFO: `/v3/services` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:22.481831 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753ecf6b-e686-4a33-a426-80a3d24da350 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:22.483169 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753ecf6b-e686-4a33-a426-80a3d24da350 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:22.488630 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 453/903] 10.222.0.58 () {56 vars in 1087 bytes} [Tue May 17 19:11:22 2022] GET /identity/v3/services => generated 1268 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:11:22.489393 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753ecf6b-e686-4a33-a426-80a3d24da350 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:22.522805 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 451/904] 10.222.0.58 () {60 vars in 1141 bytes} [Tue May 17 19:11:22 2022] POST /identity/v3/services => generated 241 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:11:22.522805 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d80d37-5716-414a-a05a-6516200c3670 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:22.522805 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d80d37-5716-414a-a05a-6516200c3670 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:22.522805 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d80d37-5716-414a-a05a-6516200c3670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:22.549324 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-14d80d37-5716-414a-a05a-6516200c3670 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:11:22.551567 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-14d80d37-5716-414a-a05a-6516200c3670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:11:22.611961 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753ecf6b-e686-4a33-a426-80a3d24da350 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:22.663519 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4740ad-443f-4f41-9c58-23bd935d346a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:22.663519 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4740ad-443f-4f41-9c58-23bd935d346a None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:22.663519 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4740ad-443f-4f41-9c58-23bd935d346a None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:22.663519 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4740ad-443f-4f41-9c58-23bd935d346a None admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:22.663962 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4740ad-443f-4f41-9c58-23bd935d346a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:22.663962 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4740ad-443f-4f41-9c58-23bd935d346a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:22.663962 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4740ad-443f-4f41-9c58-23bd935d346a None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:22.664682 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 452/905] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:11:22 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 17 19:11:22.676798 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 454/906] 10.222.0.58 () {56 vars in 1148 bytes} [Tue May 17 19:11:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:22.683122 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d80d37-5716-414a-a05a-6516200c3670 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:22.685481 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4740ad-443f-4f41-9c58-23bd935d346a None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:22.805987 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d80d37-5716-414a-a05a-6516200c3670 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:22.809586 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4740ad-443f-4f41-9c58-23bd935d346a None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:22.845155 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec04548f-c968-449f-989a-5aad66cb94a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:22.846946 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec04548f-c968-449f-989a-5aad66cb94a5 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:22.847314 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec04548f-c968-449f-989a-5aad66cb94a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:22.847711 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec04548f-c968-449f-989a-5aad66cb94a5 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:22.848097 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a0e4a2-accd-498c-8b97-fc2aa97b2d09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:22.848554 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec04548f-c968-449f-989a-5aad66cb94a5 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:22.849074 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a0e4a2-accd-498c-8b97-fc2aa97b2d09 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:22.849490 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a0e4a2-accd-498c-8b97-fc2aa97b2d09 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:22.849911 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a0e4a2-accd-498c-8b97-fc2aa97b2d09 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:22.850287 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec04548f-c968-449f-989a-5aad66cb94a5 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:22.851044 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a0e4a2-accd-498c-8b97-fc2aa97b2d09 None admin] RBAC: Adding query filter params (role.id=897862af1c83498f86d0dfdf48bea276, scope.project.id=1a9d3a0631284c9fb113266036993e2d, user.id=2b90b57c542440448579b3aadda8b241) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:22.851504 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a0e4a2-accd-498c-8b97-fc2aa97b2d09 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:22.853567 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a0e4a2-accd-498c-8b97-fc2aa97b2d09 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:22.860089 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 453/907] 10.222.0.58 () {56 vars in 1374 bytes} [Tue May 17 19:11:22 2022] GET /identity/v3/role_assignments?user.id=2b90b57c542440448579b3aadda8b241&scope.project.id=1a9d3a0631284c9fb113266036993e2d&role.id=897862af1c83498f86d0dfdf48bea276 => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:11:22.867272 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-ec04548f-c968-449f-989a-5aad66cb94a5 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 17 19:11:22.867272 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:22.867272 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 17 19:11:22.867272 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 17 19:11:22.867272 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 17 19:11:22.867272 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 17 19:11:22.867272 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 17 19:11:22.867272 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 17 19:11:22.867272 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 17 19:11:22.867272 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:11:22.867272 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 17 19:11:22.867272 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:11:22.867272 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:22.867272 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:11:22.867272 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:11:22.867272 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:11:22.868464 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:11:22.868464 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:11:22.868464 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:11:22.868464 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:11:22.868464 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:11:22.868464 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:11:22.868464 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:11:22.868464 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 17 19:11:22.868464 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_service(service_id) May 17 19:11:22.868464 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 17 19:11:22.868464 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 17 19:11:22.868464 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:11:22.868464 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:11:22.868464 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:11:22.868464 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:11:22.868464 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:11:22.869502 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:11:22.869502 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:11:22.869502 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:11:22.869502 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:11:22.869502 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:11:22.869502 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:11:22.869502 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:11:22.869502 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:11:22.869502 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:11:22.869502 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:11:22.869502 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:11:22.869502 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 17 19:11:22.869502 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 17 19:11:22.869502 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 17 19:11:22.869502 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:11:22.871757 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 455/908] 10.222.0.58 () {56 vars in 1132 bytes} [Tue May 17 19:11:22 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:11:22.877493 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a0e4a2-accd-498c-8b97-fc2aa97b2d09 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:22.998635 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a0e4a2-accd-498c-8b97-fc2aa97b2d09 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:23.036170 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2444758b-fc44-43f5-ad7a-28ae49051f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:23.036844 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2444758b-fc44-43f5-ad7a-28ae49051f76 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:23.037119 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2444758b-fc44-43f5-ad7a-28ae49051f76 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:23.037390 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2444758b-fc44-43f5-ad7a-28ae49051f76 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:23.038102 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2444758b-fc44-43f5-ad7a-28ae49051f76 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:23.038398 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2444758b-fc44-43f5-ad7a-28ae49051f76 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:23.039899 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2444758b-fc44-43f5-ad7a-28ae49051f76 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:23.045550 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 454/909] 10.222.0.58 () {56 vars in 1126 bytes} [Tue May 17 19:11:22 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:23.051357 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec04548f-c968-449f-989a-5aad66cb94a5 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:23.154109 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec04548f-c968-449f-989a-5aad66cb94a5 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:23.187036 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c9ac82-2998-4596-b987-3ff634c222b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:23.188031 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c9ac82-2998-4596-b987-3ff634c222b1 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:23.188502 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c9ac82-2998-4596-b987-3ff634c222b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:23.188945 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c9ac82-2998-4596-b987-3ff634c222b1 None admin] PATH_INFO: `/v3/services` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:23.189844 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c9ac82-2998-4596-b987-3ff634c222b1 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:23.190323 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c9ac82-2998-4596-b987-3ff634c222b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:23.191856 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c9ac82-2998-4596-b987-3ff634c222b1 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:23.196984 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 456/910] 10.222.0.58 () {56 vars in 1126 bytes} [Tue May 17 19:11:23 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:23.202779 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2444758b-fc44-43f5-ad7a-28ae49051f76 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:23.302406 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2444758b-fc44-43f5-ad7a-28ae49051f76 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:23.336975 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef882d3b-d1d8-4b07-a138-69044eaa1a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:23.337923 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef882d3b-d1d8-4b07-a138-69044eaa1a1d None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:23.338478 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef882d3b-d1d8-4b07-a138-69044eaa1a1d None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:23.338923 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef882d3b-d1d8-4b07-a138-69044eaa1a1d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:23.339898 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef882d3b-d1d8-4b07-a138-69044eaa1a1d None admin] RBAC: Adding query filter params (interface=public, service_id=89b948cc07ee4aceae51e944d5740c70, region_id=RegionOne) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:23.340377 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef882d3b-d1d8-4b07-a138-69044eaa1a1d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:23.341897 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef882d3b-d1d8-4b07-a138-69044eaa1a1d None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:23.347966 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 455/911] 10.222.0.58 () {56 vars in 1251 bytes} [Tue May 17 19:11:23 2022] GET /identity/v3/endpoints?service_id=89b948cc07ee4aceae51e944d5740c70&interface=public®ion_id=RegionOne => generated 194 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:23.353542 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c9ac82-2998-4596-b987-3ff634c222b1 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:23.455824 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c9ac82-2998-4596-b987-3ff634c222b1 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:23.489823 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7d174d-6dde-439e-877f-cef1f12280ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:23.490740 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7d174d-6dde-439e-877f-cef1f12280ad None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:23.491215 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7d174d-6dde-439e-877f-cef1f12280ad None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:23.491686 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7d174d-6dde-439e-877f-cef1f12280ad None admin] PATH_INFO: `/v3/services` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:23.492581 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7d174d-6dde-439e-877f-cef1f12280ad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:23.494160 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7d174d-6dde-439e-877f-cef1f12280ad None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:23.500455 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 457/912] 10.222.0.58 () {56 vars in 1087 bytes} [Tue May 17 19:11:23 2022] GET /identity/v3/services => generated 1497 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:11:24.775465 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cdd82b-2fd8-4bee-bb5e-491bde4caf56 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:24.776004 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cdd82b-2fd8-4bee-bb5e-491bde4caf56 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:24.776004 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cdd82b-2fd8-4bee-bb5e-491bde4caf56 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:24.777574 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 456/913] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:11:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:11:24.782948 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c917be-3e05-41fb-a9d5-e1f79ef7b090 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:24.783385 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c917be-3e05-41fb-a9d5-e1f79ef7b090 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:24.783619 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c917be-3e05-41fb-a9d5-e1f79ef7b090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:24.794987 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dccb06-9c5c-4409-85c8-01904cd51203 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:24.795281 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dccb06-9c5c-4409-85c8-01904cd51203 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:24.795540 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dccb06-9c5c-4409-85c8-01904cd51203 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:24.797123 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 457/914] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:11:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 17 19:11:24.803830 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54e8e0a-5db1-438d-a8a0-192362d1a39a None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:24.804380 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54e8e0a-5db1-438d-a8a0-192362d1a39a None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:24.804488 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-00c917be-3e05-41fb-a9d5-e1f79ef7b090 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:11:24.804969 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54e8e0a-5db1-438d-a8a0-192362d1a39a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:24.805634 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-00c917be-3e05-41fb-a9d5-e1f79ef7b090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:11:24.822232 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-b54e8e0a-5db1-438d-a8a0-192362d1a39a None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:11:24.823732 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b54e8e0a-5db1-438d-a8a0-192362d1a39a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:11:24.873016 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 458/915] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:11:24 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 17 19:11:24.895023 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c917be-3e05-41fb-a9d5-e1f79ef7b090 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:24.900619 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 458/916] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:11:24 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 17 19:11:24.923133 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54e8e0a-5db1-438d-a8a0-192362d1a39a None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:24.995336 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c917be-3e05-41fb-a9d5-e1f79ef7b090 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:25.029914 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f10a48-1141-4209-a299-0d72564105ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:25.030689 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f10a48-1141-4209-a299-0d72564105ad None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:25.030971 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f10a48-1141-4209-a299-0d72564105ad None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:25.031241 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f10a48-1141-4209-a299-0d72564105ad None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:25.031913 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f10a48-1141-4209-a299-0d72564105ad None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:25.032358 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54e8e0a-5db1-438d-a8a0-192362d1a39a None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:25.033265 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f10a48-1141-4209-a299-0d72564105ad None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:25.048911 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-73f10a48-1141-4209-a299-0d72564105ad None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 17 19:11:25.048911 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:25.048911 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 17 19:11:25.048911 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 17 19:11:25.048911 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 17 19:11:25.048911 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 17 19:11:25.048911 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 17 19:11:25.048911 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 17 19:11:25.048911 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 17 19:11:25.048911 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:11:25.048911 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 17 19:11:25.048911 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:11:25.048911 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:25.048911 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:11:25.048911 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:11:25.048911 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:11:25.048911 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:11:25.049992 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:11:25.049992 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:11:25.049992 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:11:25.049992 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:11:25.049992 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:11:25.049992 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:11:25.049992 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 17 19:11:25.049992 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_service(service_id) May 17 19:11:25.049992 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 17 19:11:25.049992 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 17 19:11:25.049992 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:11:25.049992 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:11:25.049992 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:11:25.049992 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:11:25.049992 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:11:25.049992 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:11:25.050831 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:11:25.050831 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:11:25.050831 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:11:25.050831 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:11:25.050831 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:11:25.050831 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:11:25.050831 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:11:25.050831 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:11:25.050831 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:11:25.050831 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:11:25.050831 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 17 19:11:25.050831 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 17 19:11:25.050831 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 17 19:11:25.050831 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:11:25.055299 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 459/917] 10.222.0.58 () {56 vars in 1126 bytes} [Tue May 17 19:11:24 2022] GET /identity/v3/services/object-store => generated 93 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 17 19:11:25.056198 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f10a48-1141-4209-a299-0d72564105ad None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:25.074018 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac57813-7047-46d3-9f6c-b0332476eec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:25.075128 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac57813-7047-46d3-9f6c-b0332476eec2 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:25.075655 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac57813-7047-46d3-9f6c-b0332476eec2 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:25.076149 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac57813-7047-46d3-9f6c-b0332476eec2 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:25.077000 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac57813-7047-46d3-9f6c-b0332476eec2 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:25.078583 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac57813-7047-46d3-9f6c-b0332476eec2 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:25.090486 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-7ac57813-7047-46d3-9f6c-b0332476eec2 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 17 19:11:25.090486 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:25.090486 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 17 19:11:25.090486 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 17 19:11:25.090486 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 17 19:11:25.090486 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 17 19:11:25.090486 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 17 19:11:25.090486 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 17 19:11:25.090486 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 17 19:11:25.090486 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:11:25.090486 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 17 19:11:25.090486 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:11:25.090486 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:25.090486 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:11:25.090486 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:11:25.090486 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:11:25.090486 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:11:25.091470 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:11:25.091470 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:11:25.091470 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:11:25.091470 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:11:25.091470 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:11:25.091470 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:11:25.091470 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 17 19:11:25.091470 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(service_id) May 17 19:11:25.091470 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 17 19:11:25.091470 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 17 19:11:25.091470 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:11:25.091470 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:11:25.091470 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:11:25.091470 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:11:25.091470 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:11:25.091470 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:11:25.092329 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:11:25.092329 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:11:25.092329 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:11:25.092329 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:11:25.092329 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:11:25.092329 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:11:25.092329 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:11:25.092329 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:11:25.092329 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:11:25.092329 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:11:25.092329 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 17 19:11:25.092329 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 17 19:11:25.092329 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 17 19:11:25.092329 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:11:25.100590 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 459/918] 10.222.0.58 () {56 vars in 1114 bytes} [Tue May 17 19:11:24 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:11:25.101678 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac57813-7047-46d3-9f6c-b0332476eec2 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:25.174273 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f10a48-1141-4209-a299-0d72564105ad None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:25.202762 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac57813-7047-46d3-9f6c-b0332476eec2 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:25.214976 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87caeb5-3999-4e98-8b96-5b672a2d869a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:25.215825 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87caeb5-3999-4e98-8b96-5b672a2d869a None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:25.216105 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87caeb5-3999-4e98-8b96-5b672a2d869a None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:25.216749 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87caeb5-3999-4e98-8b96-5b672a2d869a None admin] PATH_INFO: `/v3/services` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:25.217724 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87caeb5-3999-4e98-8b96-5b672a2d869a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:25.218031 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87caeb5-3999-4e98-8b96-5b672a2d869a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:25.219854 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87caeb5-3999-4e98-8b96-5b672a2d869a None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:25.225276 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 460/919] 10.222.0.58 () {56 vars in 1122 bytes} [Tue May 17 19:11:25 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:25.226732 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0499c4c6-fd3a-49c5-91b9-f40ce8d3800a None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:25.227019 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0499c4c6-fd3a-49c5-91b9-f40ce8d3800a None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:25.227297 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0499c4c6-fd3a-49c5-91b9-f40ce8d3800a None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:25.228672 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 461/920] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:11:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:11:25.230651 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0499c4c6-fd3a-49c5-91b9-f40ce8d3800a None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:25.242150 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be8d515-b22b-4d4f-b69e-3e4aa6b3fac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:25.242818 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be8d515-b22b-4d4f-b69e-3e4aa6b3fac8 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:25.243290 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be8d515-b22b-4d4f-b69e-3e4aa6b3fac8 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:25.243397 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be8d515-b22b-4d4f-b69e-3e4aa6b3fac8 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:25.244089 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be8d515-b22b-4d4f-b69e-3e4aa6b3fac8 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:25.244393 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be8d515-b22b-4d4f-b69e-3e4aa6b3fac8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:25.245765 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be8d515-b22b-4d4f-b69e-3e4aa6b3fac8 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:25.250789 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 460/921] 10.222.0.58 () {56 vars in 1114 bytes} [Tue May 17 19:11:25 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:25.252090 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3d7345-6105-4679-817b-dddffd2360bc None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:25.252896 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3d7345-6105-4679-817b-dddffd2360bc None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:25.253169 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3d7345-6105-4679-817b-dddffd2360bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:25.272416 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-0f3d7345-6105-4679-817b-dddffd2360bc None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:11:25.273585 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-0f3d7345-6105-4679-817b-dddffd2360bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:11:25.333465 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0499c4c6-fd3a-49c5-91b9-f40ce8d3800a None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:25.351878 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 461/922] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:11:25 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 17 19:11:25.352892 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3d7345-6105-4679-817b-dddffd2360bc None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:25.368926 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2729beef-ce43-47b0-a562-bd167b3b45a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:25.369592 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2729beef-ce43-47b0-a562-bd167b3b45a8 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:25.369920 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2729beef-ce43-47b0-a562-bd167b3b45a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:25.370197 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2729beef-ce43-47b0-a562-bd167b3b45a8 None admin] PATH_INFO: `/v3/services` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:25.370928 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2729beef-ce43-47b0-a562-bd167b3b45a8 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:25.371229 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2729beef-ce43-47b0-a562-bd167b3b45a8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:25.372627 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2729beef-ce43-47b0-a562-bd167b3b45a8 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:25.377857 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 462/923] 10.222.0.58 () {56 vars in 1122 bytes} [Tue May 17 19:11:25 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:25.378716 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2729beef-ce43-47b0-a562-bd167b3b45a8 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:25.458058 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3d7345-6105-4679-817b-dddffd2360bc None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:25.474984 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2729beef-ce43-47b0-a562-bd167b3b45a8 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:25.492866 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d11b2f-b382-451e-8848-3aa64340e4d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:25.493522 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d11b2f-b382-451e-8848-3aa64340e4d5 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:25.493796 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d11b2f-b382-451e-8848-3aa64340e4d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:25.494124 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d11b2f-b382-451e-8848-3aa64340e4d5 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:25.494816 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d11b2f-b382-451e-8848-3aa64340e4d5 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:25.495167 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d11b2f-b382-451e-8848-3aa64340e4d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:25.496617 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d11b2f-b382-451e-8848-3aa64340e4d5 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:25.501614 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 462/924] 10.222.0.58 () {56 vars in 1114 bytes} [Tue May 17 19:11:25 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:25.503016 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d11b2f-b382-451e-8848-3aa64340e4d5 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:25.512907 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ad199d-1b0b-44c2-8fee-648205c53454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:25.513717 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ad199d-1b0b-44c2-8fee-648205c53454 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:25.513999 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ad199d-1b0b-44c2-8fee-648205c53454 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:25.514273 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ad199d-1b0b-44c2-8fee-648205c53454 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:25.524348 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ad199d-1b0b-44c2-8fee-648205c53454 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 17 19:11:25.532741 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-29ad199d-1b0b-44c2-8fee-648205c53454 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 17 19:11:25.532741 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:25.532741 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:11:25.532741 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:11:25.532741 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:11:25.532741 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:11:25.532741 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:11:25.532741 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:11:25.532741 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:11:25.532741 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:11:25.532741 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:11:25.532741 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:11:25.532741 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 17 19:11:25.532741 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) May 17 19:11:25.532741 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 17 19:11:25.532741 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 17 19:11:25.533905 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 17 19:11:25.533905 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 17 19:11:25.533905 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 17 19:11:25.533905 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 17 19:11:25.533905 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:11:25.533905 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:11:25.533905 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:11:25.533905 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:11:25.533905 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:11:25.533905 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:11:25.533905 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:11:25.533905 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:11:25.533905 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:11:25.533905 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:11:25.533905 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:11:25.533905 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:11:25.534767 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:11:25.534767 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:11:25.534767 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:11:25.534767 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:11:25.534767 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 17 19:11:25.534767 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 17 19:11:25.534767 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 17 19:11:25.534767 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 17 19:11:25.534767 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 17 19:11:25.534767 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 17 19:11:25.534767 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 17 19:11:25.534767 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:11:25.534767 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 463/925] 10.222.0.58 () {56 vars in 1120 bytes} [Tue May 17 19:11:25 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:11:25.535460 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ad199d-1b0b-44c2-8fee-648205c53454 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:25.610514 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d11b2f-b382-451e-8848-3aa64340e4d5 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:25.637963 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ad199d-1b0b-44c2-8fee-648205c53454 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:25.646227 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c043283a-36cd-4b21-b57f-9cc1daadb63d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:25.650482 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c043283a-36cd-4b21-b57f-9cc1daadb63d None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:25.650482 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c043283a-36cd-4b21-b57f-9cc1daadb63d None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:25.650482 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c043283a-36cd-4b21-b57f-9cc1daadb63d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:25.650482 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c043283a-36cd-4b21-b57f-9cc1daadb63d None admin] RBAC: Adding query filter params (interface=public, service_id=948e60b6a47d46a6a5e964b5e3260b73, region_id=RegionOne) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:25.650482 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c043283a-36cd-4b21-b57f-9cc1daadb63d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:25.650482 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c043283a-36cd-4b21-b57f-9cc1daadb63d None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:25.655781 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 463/926] 10.222.0.58 () {56 vars in 1251 bytes} [Tue May 17 19:11:25 2022] GET /identity/v3/endpoints?service_id=948e60b6a47d46a6a5e964b5e3260b73&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:25.656005 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c043283a-36cd-4b21-b57f-9cc1daadb63d None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:25.672450 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d6e5ca-b5c6-4944-9271-cdfa91332407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:25.673102 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d6e5ca-b5c6-4944-9271-cdfa91332407 None admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:25.673377 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d6e5ca-b5c6-4944-9271-cdfa91332407 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:25.673645 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d6e5ca-b5c6-4944-9271-cdfa91332407 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:25.674982 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d6e5ca-b5c6-4944-9271-cdfa91332407 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:25.676366 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d6e5ca-b5c6-4944-9271-cdfa91332407 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:25.757312 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c043283a-36cd-4b21-b57f-9cc1daadb63d None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:25.789115 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ac9c46-1e1a-4ad8-851a-ab927761bc32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:25.789757 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ac9c46-1e1a-4ad8-851a-ab927761bc32 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:25.790031 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ac9c46-1e1a-4ad8-851a-ab927761bc32 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:25.790299 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ac9c46-1e1a-4ad8-851a-ab927761bc32 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:25.791026 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ac9c46-1e1a-4ad8-851a-ab927761bc32 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:25.791322 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ac9c46-1e1a-4ad8-851a-ab927761bc32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:25.792713 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ac9c46-1e1a-4ad8-851a-ab927761bc32 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:25.796077 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 464/927] 10.222.0.58 () {60 vars in 1144 bytes} [Tue May 17 19:11:25 2022] POST /identity/v3/endpoints => generated 346 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:11:25.799854 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d6e5ca-b5c6-4944-9271-cdfa91332407 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:25.801117 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 464/928] 10.222.0.58 () {56 vars in 1115 bytes} [Tue May 17 19:11:25 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:25.802440 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38004c41-68a7-4c18-b97f-4e11ab5e0b29 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:25.802717 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38004c41-68a7-4c18-b97f-4e11ab5e0b29 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:25.802985 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38004c41-68a7-4c18-b97f-4e11ab5e0b29 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:25.804223 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 465/929] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:11:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:11:25.809278 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38004c41-68a7-4c18-b97f-4e11ab5e0b29 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:25.914896 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d6e5ca-b5c6-4944-9271-cdfa91332407 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:25.926767 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38004c41-68a7-4c18-b97f-4e11ab5e0b29 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:25.956672 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf1976f-d2a5-4c0e-a600-a44e7cdff6a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:25.957345 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf1976f-d2a5-4c0e-a600-a44e7cdff6a7 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:25.957626 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf1976f-d2a5-4c0e-a600-a44e7cdff6a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:25.957900 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf1976f-d2a5-4c0e-a600-a44e7cdff6a7 None admin] PATH_INFO: `/v3/services` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:25.958585 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf1976f-d2a5-4c0e-a600-a44e7cdff6a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:25.960058 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf1976f-d2a5-4c0e-a600-a44e7cdff6a7 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:25.966371 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 465/930] 10.222.0.58 () {56 vars in 1087 bytes} [Tue May 17 19:11:25 2022] GET /identity/v3/services => generated 1497 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:11:25.967703 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd2c972-f2d6-4233-aea5-7369142fdd4a None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:25.967969 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd2c972-f2d6-4233-aea5-7369142fdd4a None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:25.968240 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd2c972-f2d6-4233-aea5-7369142fdd4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:25.976490 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f90b7d-ea1c-4eea-9570-51f7c9095a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:25.977305 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f90b7d-ea1c-4eea-9570-51f7c9095a6c None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:25.977580 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f90b7d-ea1c-4eea-9570-51f7c9095a6c None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:25.977848 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f90b7d-ea1c-4eea-9570-51f7c9095a6c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:25.992240 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-3fd2c972-f2d6-4233-aea5-7369142fdd4a None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:11:25.992973 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-3fd2c972-f2d6-4233-aea5-7369142fdd4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:11:25.994076 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f90b7d-ea1c-4eea-9570-51f7c9095a6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:25.995693 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f90b7d-ea1c-4eea-9570-51f7c9095a6c None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:26.018360 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-90f90b7d-ea1c-4eea-9570-51f7c9095a6c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:11:26.018360 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:26.018360 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:11:26.018360 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:11:26.018360 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:11:26.018360 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:11:26.018360 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:11:26.018360 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:11:26.018360 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:11:26.018360 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:11:26.018360 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:11:26.018360 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:11:26.018360 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 17 19:11:26.018360 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 17 19:11:26.018360 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 17 19:11:26.019366 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 17 19:11:26.019366 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:11:26.019366 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:11:26.019366 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:11:26.019366 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:11:26.019366 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:11:26.019366 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:11:26.019366 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:11:26.019366 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:11:26.019366 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:11:26.019366 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:11:26.019366 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:11:26.019366 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:11:26.019366 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:11:26.019366 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:11:26.019366 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:11:26.019366 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:11:26.024615 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 17 19:11:26.024615 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 17 19:11:26.024615 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:11:26.024615 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:11:26.024615 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 466/931] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:11:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:11:26.027079 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f90b7d-ea1c-4eea-9570-51f7c9095a6c None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:26.086752 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 466/932] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:11:25 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 17 19:11:26.108571 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd2c972-f2d6-4233-aea5-7369142fdd4a None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:26.152793 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f90b7d-ea1c-4eea-9570-51f7c9095a6c None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:26.200779 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d257b7-d12f-47fe-a86e-56fd6b8aab81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:26.201767 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d257b7-d12f-47fe-a86e-56fd6b8aab81 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:26.202238 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d257b7-d12f-47fe-a86e-56fd6b8aab81 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:26.202701 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d257b7-d12f-47fe-a86e-56fd6b8aab81 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:26.204073 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d257b7-d12f-47fe-a86e-56fd6b8aab81 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:26.204583 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d257b7-d12f-47fe-a86e-56fd6b8aab81 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:26.207023 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d257b7-d12f-47fe-a86e-56fd6b8aab81 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:26.224182 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 467/933] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 17 19:11:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:26.229680 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d257b7-d12f-47fe-a86e-56fd6b8aab81 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:26.233927 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd2c972-f2d6-4233-aea5-7369142fdd4a None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:26.275527 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4985b661-99cb-4263-a4ae-ebd6924d340b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:26.276326 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4985b661-99cb-4263-a4ae-ebd6924d340b None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:26.276606 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4985b661-99cb-4263-a4ae-ebd6924d340b None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:26.276881 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4985b661-99cb-4263-a4ae-ebd6924d340b None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:26.277537 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4985b661-99cb-4263-a4ae-ebd6924d340b None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:26.279350 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4985b661-99cb-4263-a4ae-ebd6924d340b None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:26.296715 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-4985b661-99cb-4263-a4ae-ebd6924d340b None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 17 19:11:26.296715 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:26.296715 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 17 19:11:26.296715 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 17 19:11:26.296715 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 17 19:11:26.296715 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 17 19:11:26.296715 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 17 19:11:26.296715 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 17 19:11:26.296715 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 17 19:11:26.296715 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:11:26.296715 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 17 19:11:26.296715 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:11:26.296715 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:26.296715 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:11:26.296715 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:11:26.296715 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:11:26.297734 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:11:26.297734 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:11:26.297734 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:11:26.297734 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:11:26.297734 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:11:26.297734 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:11:26.297734 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:11:26.297734 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 17 19:11:26.297734 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_service(service_id) May 17 19:11:26.297734 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 17 19:11:26.297734 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 17 19:11:26.297734 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:11:26.297734 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:11:26.297734 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:11:26.297734 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:11:26.297734 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:11:26.299776 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:11:26.299776 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:11:26.299776 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:11:26.299776 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:11:26.299776 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:11:26.299776 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:11:26.299776 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:11:26.299776 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:11:26.299776 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:11:26.299776 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:11:26.299776 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:11:26.299776 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 17 19:11:26.299776 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 17 19:11:26.299776 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 17 19:11:26.299776 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:11:26.299776 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 467/934] 10.222.0.58 () {56 vars in 1132 bytes} [Tue May 17 19:11:26 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 17 19:11:26.304051 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4985b661-99cb-4263-a4ae-ebd6924d340b None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:26.343839 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d257b7-d12f-47fe-a86e-56fd6b8aab81 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:26.380614 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710dd6e7-5d39-490a-9962-1da1cd3e41aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:26.381481 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710dd6e7-5d39-490a-9962-1da1cd3e41aa None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:26.381766 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710dd6e7-5d39-490a-9962-1da1cd3e41aa None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:26.382037 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710dd6e7-5d39-490a-9962-1da1cd3e41aa None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:26.389538 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710dd6e7-5d39-490a-9962-1da1cd3e41aa None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:26.391049 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710dd6e7-5d39-490a-9962-1da1cd3e41aa None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:26.406582 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-710dd6e7-5d39-490a-9962-1da1cd3e41aa None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 17 19:11:26.406582 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:26.406582 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:11:26.406582 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:11:26.406582 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:11:26.406582 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:11:26.406582 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:11:26.406582 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:11:26.406582 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:11:26.406582 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:11:26.406582 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:11:26.406582 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:11:26.406582 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 17 19:11:26.406582 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_user(user_id) May 17 19:11:26.406582 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 17 19:11:26.406582 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 17 19:11:26.407716 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:11:26.407716 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:11:26.407716 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 17 19:11:26.407716 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 17 19:11:26.407716 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 17 19:11:26.407716 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 17 19:11:26.407716 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:11:26.407716 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:11:26.407716 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:11:26.407716 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:11:26.407716 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:11:26.407716 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:11:26.407716 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:11:26.407716 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:11:26.407716 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:11:26.407716 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:11:26.407716 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:11:26.407716 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:11:26.408730 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:11:26.408730 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:11:26.408730 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 17 19:11:26.408730 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 17 19:11:26.408730 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 17 19:11:26.408730 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 17 19:11:26.408730 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 17 19:11:26.408730 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 17 19:11:26.408730 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 17 19:11:26.408730 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:11:26.408730 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 468/935] 10.222.0.58 () {56 vars in 1117 bytes} [Tue May 17 19:11:26 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:11:26.416801 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710dd6e7-5d39-490a-9962-1da1cd3e41aa None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:26.429454 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4985b661-99cb-4263-a4ae-ebd6924d340b None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:26.470207 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0754db0b-98c0-4845-bb8f-3d8550c4d93a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:26.470792 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0754db0b-98c0-4845-bb8f-3d8550c4d93a None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:26.471638 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0754db0b-98c0-4845-bb8f-3d8550c4d93a None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:26.471911 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0754db0b-98c0-4845-bb8f-3d8550c4d93a None admin] PATH_INFO: `/v3/services` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:26.472790 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0754db0b-98c0-4845-bb8f-3d8550c4d93a None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:26.473096 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0754db0b-98c0-4845-bb8f-3d8550c4d93a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:26.475112 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0754db0b-98c0-4845-bb8f-3d8550c4d93a None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:26.481102 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 468/936] 10.222.0.58 () {56 vars in 1126 bytes} [Tue May 17 19:11:26 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:11:26.485706 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0754db0b-98c0-4845-bb8f-3d8550c4d93a None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:26.535679 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710dd6e7-5d39-490a-9962-1da1cd3e41aa None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:26.570377 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda04ade-24ca-4fa9-89e7-30d654286d70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:26.571139 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda04ade-24ca-4fa9-89e7-30d654286d70 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:26.571514 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda04ade-24ca-4fa9-89e7-30d654286d70 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:26.571858 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda04ade-24ca-4fa9-89e7-30d654286d70 None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:26.572930 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda04ade-24ca-4fa9-89e7-30d654286d70 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:26.573050 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda04ade-24ca-4fa9-89e7-30d654286d70 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:26.574660 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda04ade-24ca-4fa9-89e7-30d654286d70 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:26.588696 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0754db0b-98c0-4845-bb8f-3d8550c4d93a None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:26.592820 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 469/937] 10.222.0.58 () {56 vars in 1149 bytes} [Tue May 17 19:11:26 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:26.598732 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda04ade-24ca-4fa9-89e7-30d654286d70 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:26.626930 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fe6613-5758-4ff2-b12d-02f5ffda1acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:26.628086 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fe6613-5758-4ff2-b12d-02f5ffda1acc None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:26.628693 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fe6613-5758-4ff2-b12d-02f5ffda1acc None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:26.629524 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fe6613-5758-4ff2-b12d-02f5ffda1acc None admin] PATH_INFO: `/v3/services` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:26.630676 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fe6613-5758-4ff2-b12d-02f5ffda1acc None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:26.631152 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fe6613-5758-4ff2-b12d-02f5ffda1acc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:26.633477 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fe6613-5758-4ff2-b12d-02f5ffda1acc None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:26.640886 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 469/938] 10.222.0.58 () {56 vars in 1126 bytes} [Tue May 17 19:11:26 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:11:26.646367 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fe6613-5758-4ff2-b12d-02f5ffda1acc None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:26.723227 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda04ade-24ca-4fa9-89e7-30d654286d70 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:26.758285 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fe6613-5758-4ff2-b12d-02f5ffda1acc None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:26.759386 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0ce4cc-1fdc-45cc-b348-dfbe23774ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:26.760415 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0ce4cc-1fdc-45cc-b348-dfbe23774ec7 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:26.760891 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0ce4cc-1fdc-45cc-b348-dfbe23774ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:26.761364 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0ce4cc-1fdc-45cc-b348-dfbe23774ec7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:26.771835 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0ce4cc-1fdc-45cc-b348-dfbe23774ec7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:26.773515 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0ce4cc-1fdc-45cc-b348-dfbe23774ec7 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:26.791977 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-7e0ce4cc-1fdc-45cc-b348-dfbe23774ec7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:11:26.791977 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:26.791977 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:11:26.791977 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:11:26.791977 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:11:26.791977 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:11:26.791977 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:11:26.791977 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:11:26.791977 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:11:26.791977 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:11:26.791977 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:11:26.791977 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:11:26.791977 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 17 19:11:26.791977 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 17 19:11:26.791977 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 17 19:11:26.793308 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 17 19:11:26.793308 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:11:26.793308 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:11:26.793308 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:11:26.793308 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:11:26.793308 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:11:26.793308 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:11:26.793308 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:11:26.793308 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:11:26.793308 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:11:26.793308 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:11:26.793308 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:11:26.793308 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:11:26.793308 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:11:26.793308 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:11:26.793308 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:11:26.793308 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:11:26.794198 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 17 19:11:26.794198 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 17 19:11:26.794198 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:11:26.794198 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:11:26.795583 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 470/939] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:11:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:11:26.800760 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f800cc-6605-4301-b170-94583ef66846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:26.801422 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f800cc-6605-4301-b170-94583ef66846 None admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:26.801657 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0ce4cc-1fdc-45cc-b348-dfbe23774ec7 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:26.801756 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f800cc-6605-4301-b170-94583ef66846 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:26.801961 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f800cc-6605-4301-b170-94583ef66846 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:26.802799 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f800cc-6605-4301-b170-94583ef66846 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:26.804314 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f800cc-6605-4301-b170-94583ef66846 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:26.898865 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0ce4cc-1fdc-45cc-b348-dfbe23774ec7 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:26.911281 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 470/940] 10.222.0.58 () {60 vars in 1144 bytes} [Tue May 17 19:11:26 2022] POST /identity/v3/endpoints => generated 347 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 17 19:11:26.949891 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892eedad-ba4f-4f84-85f8-4fe4be45d49a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:26.950740 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892eedad-ba4f-4f84-85f8-4fe4be45d49a None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:26.951198 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892eedad-ba4f-4f84-85f8-4fe4be45d49a None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:26.951659 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892eedad-ba4f-4f84-85f8-4fe4be45d49a None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:26.952541 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892eedad-ba4f-4f84-85f8-4fe4be45d49a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:26.952996 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892eedad-ba4f-4f84-85f8-4fe4be45d49a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:26.954614 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892eedad-ba4f-4f84-85f8-4fe4be45d49a None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:26.968445 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 471/941] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 17 19:11:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:26.975074 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f800cc-6605-4301-b170-94583ef66846 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:27.085995 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f800cc-6605-4301-b170-94583ef66846 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:27.122628 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf1892a-5641-43f6-80fc-142a8e74d5e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:27.123441 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf1892a-5641-43f6-80fc-142a8e74d5e6 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:27.123754 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf1892a-5641-43f6-80fc-142a8e74d5e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:27.124031 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf1892a-5641-43f6-80fc-142a8e74d5e6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:27.131977 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf1892a-5641-43f6-80fc-142a8e74d5e6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:27.133470 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf1892a-5641-43f6-80fc-142a8e74d5e6 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:27.149172 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-2bf1892a-5641-43f6-80fc-142a8e74d5e6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 17 19:11:27.149172 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:27.149172 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:11:27.149172 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:11:27.149172 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:11:27.149172 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:11:27.149172 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:11:27.149172 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:11:27.149172 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:11:27.149172 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:11:27.149172 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:11:27.149172 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:11:27.149172 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 17 19:11:27.149172 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(project_id) May 17 19:11:27.149172 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 17 19:11:27.150301 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 17 19:11:27.150301 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:11:27.150301 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:11:27.150301 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:11:27.150301 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:11:27.150301 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:11:27.150301 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:11:27.150301 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:11:27.150301 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:11:27.150301 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:11:27.150301 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:11:27.150301 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:11:27.150301 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:11:27.150301 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:11:27.150301 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:11:27.150301 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:11:27.150301 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:11:27.151470 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 17 19:11:27.151470 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 17 19:11:27.151470 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 17 19:11:27.151470 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 17 19:11:27.151470 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 17 19:11:27.151470 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 17 19:11:27.151470 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 17 19:11:27.151470 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:11:27.151470 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 471/942] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 17 19:11:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 17 19:11:27.156724 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892eedad-ba4f-4f84-85f8-4fe4be45d49a None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:27.262000 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892eedad-ba4f-4f84-85f8-4fe4be45d49a None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:27.296209 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800158d4-7ec3-45b3-8b2e-6e0660b2ee55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:27.297021 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800158d4-7ec3-45b3-8b2e-6e0660b2ee55 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:27.297398 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800158d4-7ec3-45b3-8b2e-6e0660b2ee55 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:27.297767 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800158d4-7ec3-45b3-8b2e-6e0660b2ee55 None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:27.298637 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800158d4-7ec3-45b3-8b2e-6e0660b2ee55 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:27.299092 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800158d4-7ec3-45b3-8b2e-6e0660b2ee55 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:27.300688 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800158d4-7ec3-45b3-8b2e-6e0660b2ee55 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:27.313212 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 472/943] 10.222.0.58 () {56 vars in 1148 bytes} [Tue May 17 19:11:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:27.319238 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf1892a-5641-43f6-80fc-142a8e74d5e6 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:27.418758 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf1892a-5641-43f6-80fc-142a8e74d5e6 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:27.452969 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc215add-dc25-48a4-b68a-b315eaf0bc32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:27.454084 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc215add-dc25-48a4-b68a-b315eaf0bc32 None admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:27.454462 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc215add-dc25-48a4-b68a-b315eaf0bc32 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:27.454863 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc215add-dc25-48a4-b68a-b315eaf0bc32 None admin] PATH_INFO: `/v3/projects/1a9d3a0631284c9fb113266036993e2d/users/2b90b57c542440448579b3aadda8b241/roles/897862af1c83498f86d0dfdf48bea276` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:27.457239 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc215add-dc25-48a4-b68a-b315eaf0bc32 None admin] RBAC: Authorizing `identity:create_grant(project_id=1a9d3a0631284c9fb113266036993e2d, user_id=2b90b57c542440448579b3aadda8b241, role_id=897862af1c83498f86d0dfdf48bea276)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:27.458727 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc215add-dc25-48a4-b68a-b315eaf0bc32 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:27.538710 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 472/944] 10.222.0.58 () {58 vars in 1439 bytes} [Tue May 17 19:11:27 2022] PUT /identity/v3/projects/1a9d3a0631284c9fb113266036993e2d/users/2b90b57c542440448579b3aadda8b241/roles/897862af1c83498f86d0dfdf48bea276 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:11:28.152751 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac062c9-7c1c-4d37-bed8-b9aae72d0ffa None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:28.153266 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac062c9-7c1c-4d37-bed8-b9aae72d0ffa None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:28.153266 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac062c9-7c1c-4d37-bed8-b9aae72d0ffa None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:28.154873 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 473/945] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:11:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:11:28.160810 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedf76cd-f2fa-48c6-bd59-c7ae0bc38aae None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:28.161228 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedf76cd-f2fa-48c6-bd59-c7ae0bc38aae None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:28.161413 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedf76cd-f2fa-48c6-bd59-c7ae0bc38aae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:28.178270 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-fedf76cd-f2fa-48c6-bd59-c7ae0bc38aae None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:11:28.179442 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-fedf76cd-f2fa-48c6-bd59-c7ae0bc38aae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:11:28.248100 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 473/946] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:11:28 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 17 19:11:28.271497 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac062c9-7c1c-4d37-bed8-b9aae72d0ffa None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:28.275141 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7640fa20-555f-4167-9545-e5b731705dbd None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:28.275429 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7640fa20-555f-4167-9545-e5b731705dbd None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:28.275744 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7640fa20-555f-4167-9545-e5b731705dbd None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:28.277383 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 474/947] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:11:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:11:28.282648 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb98dcc-d805-4ca9-adfc-5f925ba76462 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:28.282937 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb98dcc-d805-4ca9-adfc-5f925ba76462 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:28.283251 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb98dcc-d805-4ca9-adfc-5f925ba76462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:28.301117 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-bbb98dcc-d805-4ca9-adfc-5f925ba76462 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:11:28.302301 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-bbb98dcc-d805-4ca9-adfc-5f925ba76462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:11:28.376765 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac062c9-7c1c-4d37-bed8-b9aae72d0ffa None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:28.379579 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 475/948] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:11:28 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 17 19:11:28.402148 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb98dcc-d805-4ca9-adfc-5f925ba76462 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:28.425285 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94f58b6-3a3b-4aab-a5a6-c236a42c542e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:28.426754 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94f58b6-3a3b-4aab-a5a6-c236a42c542e None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:28.427277 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94f58b6-3a3b-4aab-a5a6-c236a42c542e None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:28.427921 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94f58b6-3a3b-4aab-a5a6-c236a42c542e None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:28.429281 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94f58b6-3a3b-4aab-a5a6-c236a42c542e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:28.431536 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94f58b6-3a3b-4aab-a5a6-c236a42c542e None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:28.433372 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: WARNING keystone.server.flask.common [None req-d94f58b6-3a3b-4aab-a5a6-c236a42c542e None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 17 19:11:28.481615 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.sql.core [None req-d94f58b6-3a3b-4aab-a5a6-c236a42c542e None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 17 19:11:28.481615 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 17 19:11:28.481615 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [parameters: {'id': 'a60d493c8b504c84b9f912a67dec27cc', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 17 19:11:28.481615 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80160) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 17 19:11:28.497861 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb98dcc-d805-4ca9-adfc-5f925ba76462 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:28.504232 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-d94f58b6-3a3b-4aab-a5a6-c236a42c542e None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id a60d493c8b504c84b9f912a67dec27cc.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id a60d493c8b504c84b9f912a67dec27cc. May 17 19:11:28.504232 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:28.504232 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 17 19:11:28.504232 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self.dialect.do_execute( May 17 19:11:28.504232 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 17 19:11:28.504232 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 17 19:11:28.504232 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 17 19:11:28.504232 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application result = self._query(query) May 17 19:11:28.504232 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 17 19:11:28.504232 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application conn.query(q) May 17 19:11:28.504232 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 17 19:11:28.504232 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 17 19:11:28.504232 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 17 19:11:28.505212 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application result.read() May 17 19:11:28.505212 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 17 19:11:28.505212 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 17 19:11:28.505212 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 17 19:11:28.505212 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application packet.raise_for_error() May 17 19:11:28.505212 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 17 19:11:28.505212 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 17 19:11:28.505212 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 17 19:11:28.505212 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 17 19:11:28.505212 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 17 19:11:28.505212 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:11:28.505212 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 17 19:11:28.505212 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:11:28.505212 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:28.505212 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 17 19:11:28.505212 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 17 19:11:28.505212 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project May 17 19:11:28.505212 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return project_ref.to_dict() May 17 19:11:28.506199 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 17 19:11:28.506199 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application next(self.gen) May 17 19:11:28.506199 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 17 19:11:28.506199 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application yield resource May 17 19:11:28.506199 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 17 19:11:28.506199 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application next(self.gen) May 17 19:11:28.506199 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 17 19:11:28.506199 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self.session.rollback() May 17 19:11:28.506199 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 17 19:11:28.506199 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self.force_reraise() May 17 19:11:28.506199 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 17 19:11:28.506199 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise self.value May 17 19:11:28.506199 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 17 19:11:28.506199 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 17 19:11:28.506199 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 17 19:11:28.506199 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application session.commit() May 17 19:11:28.506199 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 17 19:11:28.507100 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 17 19:11:28.507100 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 17 19:11:28.507100 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._prepare_impl() May 17 19:11:28.507100 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 17 19:11:28.507100 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self.session.flush() May 17 19:11:28.507100 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 17 19:11:28.507100 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._flush(objects) May 17 19:11:28.507100 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 17 19:11:28.507100 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 17 19:11:28.507100 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 17 19:11:28.507100 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application compat.raise_( May 17 19:11:28.507100 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 17 19:11:28.507100 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception May 17 19:11:28.507100 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 17 19:11:28.507100 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application flush_context.execute() May 17 19:11:28.507100 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 17 19:11:28.507100 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rec.execute(self) May 17 19:11:28.508031 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 17 19:11:28.508031 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 17 19:11:28.508031 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 17 19:11:28.508031 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application _emit_insert_statements( May 17 19:11:28.508031 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 17 19:11:28.508031 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application c = connection._execute_20( May 17 19:11:28.508031 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 17 19:11:28.508031 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 17 19:11:28.508031 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 17 19:11:28.508031 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 17 19:11:28.508031 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 17 19:11:28.508031 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ret = self._execute_context( May 17 19:11:28.508031 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 17 19:11:28.508031 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 17 19:11:28.508031 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 17 19:11:28.510011 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 17 19:11:28.510011 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 17 19:11:28.510011 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception May 17 19:11:28.510011 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 17 19:11:28.510011 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self.dialect.do_execute( May 17 19:11:28.510011 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 17 19:11:28.510011 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 17 19:11:28.510011 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 17 19:11:28.510011 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application result = self._query(query) May 17 19:11:28.510011 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 17 19:11:28.510011 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application conn.query(q) May 17 19:11:28.510011 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 17 19:11:28.510011 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 17 19:11:28.510011 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 17 19:11:28.510011 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application result.read() May 17 19:11:28.510011 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 17 19:11:28.510011 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 17 19:11:28.516364 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 17 19:11:28.516364 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application packet.raise_for_error() May 17 19:11:28.516364 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 17 19:11:28.516364 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 17 19:11:28.516364 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 17 19:11:28.516364 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 17 19:11:28.516364 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 17 19:11:28.516364 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 17 19:11:28.516364 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [parameters: {'id': 'a60d493c8b504c84b9f912a67dec27cc', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 17 19:11:28.516364 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 17 19:11:28.516364 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:11:28.516364 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 17 19:11:28.516364 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:11:28.516364 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:28.516364 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project May 17 19:11:28.517255 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) May 17 19:11:28.517255 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 17 19:11:28.517255 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 17 19:11:28.517255 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. May 17 19:11:28.517255 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:11:28.517255 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 17 19:11:28.517255 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:11:28.517255 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:28.517255 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:11:28.517255 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:11:28.517255 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:11:28.517255 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:11:28.517255 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:11:28.517255 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:11:28.517255 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:11:28.517255 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:11:28.517255 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:11:28.518206 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:11:28.518206 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post May 17 19:11:28.518206 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( May 17 19:11:28.518206 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:11:28.518206 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:11:28.518206 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project May 17 19:11:28.518206 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.Conflict( May 17 19:11:28.518206 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id a60d493c8b504c84b9f912a67dec27cc. May 17 19:11:28.518206 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:11:28.518206 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 474/949] 10.222.0.58 () {60 vars in 1140 bytes} [Tue May 17 19:11:28 2022] POST /identity/v3/projects => generated 262 bytes in 238 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 17 19:11:28.518206 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94f58b6-3a3b-4aab-a5a6-c236a42c542e None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:28.544837 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae03774-1130-4863-b43a-bbb3a1f23bae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:28.545959 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae03774-1130-4863-b43a-bbb3a1f23bae None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:28.546507 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae03774-1130-4863-b43a-bbb3a1f23bae None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:28.547014 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae03774-1130-4863-b43a-bbb3a1f23bae None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:28.547933 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae03774-1130-4863-b43a-bbb3a1f23bae None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:28.550061 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae03774-1130-4863-b43a-bbb3a1f23bae None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:28.566477 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-6ae03774-1130-4863-b43a-bbb3a1f23bae None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 17 19:11:28.566477 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:28.566477 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 17 19:11:28.566477 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 17 19:11:28.566477 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 17 19:11:28.566477 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 17 19:11:28.566477 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 17 19:11:28.566477 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 17 19:11:28.566477 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 17 19:11:28.566477 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:11:28.566477 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 17 19:11:28.566477 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:11:28.566477 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:28.566477 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:11:28.566477 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:11:28.566477 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:11:28.566477 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:11:28.567530 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:11:28.567530 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:11:28.567530 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:11:28.567530 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:11:28.567530 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:11:28.567530 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:11:28.567530 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 17 19:11:28.567530 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_service(service_id) May 17 19:11:28.567530 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 17 19:11:28.567530 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 17 19:11:28.567530 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:11:28.567530 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:11:28.567530 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:11:28.567530 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:11:28.567530 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:11:28.567530 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:11:28.568399 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:11:28.568399 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:11:28.568399 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:11:28.568399 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:11:28.568399 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:11:28.568399 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:11:28.568399 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:11:28.568399 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:11:28.568399 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:11:28.568399 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:11:28.568399 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 17 19:11:28.568399 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 17 19:11:28.568399 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 17 19:11:28.568399 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:11:28.571519 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 476/950] 10.222.0.58 () {56 vars in 1126 bytes} [Tue May 17 19:11:28 2022] GET /identity/v3/services/object-store => generated 93 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 17 19:11:28.578460 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae03774-1130-4863-b43a-bbb3a1f23bae None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:28.633876 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94f58b6-3a3b-4aab-a5a6-c236a42c542e None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:28.669877 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcfd18a-7879-4e55-8e75-36d4518ba845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:28.671295 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcfd18a-7879-4e55-8e75-36d4518ba845 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:28.671295 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcfd18a-7879-4e55-8e75-36d4518ba845 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:28.671295 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcfd18a-7879-4e55-8e75-36d4518ba845 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:28.679248 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcfd18a-7879-4e55-8e75-36d4518ba845 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:28.680955 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcfd18a-7879-4e55-8e75-36d4518ba845 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:28.696951 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-bbcfd18a-7879-4e55-8e75-36d4518ba845 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 17 19:11:28.696951 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:28.696951 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:11:28.696951 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:11:28.696951 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:11:28.696951 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:11:28.696951 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:11:28.696951 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:11:28.696951 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:11:28.696951 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:11:28.696951 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:11:28.696951 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:11:28.696951 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 17 19:11:28.696951 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(project_id) May 17 19:11:28.696951 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 17 19:11:28.697928 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 17 19:11:28.697928 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:11:28.697928 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:11:28.697928 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:11:28.697928 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:11:28.697928 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:11:28.697928 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:11:28.697928 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:11:28.697928 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:11:28.697928 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:11:28.697928 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:11:28.697928 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:11:28.697928 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:11:28.697928 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:11:28.697928 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:11:28.697928 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:11:28.697928 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:11:28.699570 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 17 19:11:28.699570 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 17 19:11:28.699570 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 17 19:11:28.699570 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 17 19:11:28.699570 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 17 19:11:28.699570 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 17 19:11:28.699570 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 17 19:11:28.699570 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:11:28.703333 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 475/951] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 17 19:11:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 17 19:11:28.704320 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcfd18a-7879-4e55-8e75-36d4518ba845 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:28.710246 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae03774-1130-4863-b43a-bbb3a1f23bae None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:28.750996 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4d96c5-acba-413c-a429-8c9eaa67fe0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:28.751709 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4d96c5-acba-413c-a429-8c9eaa67fe0d None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:28.752516 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4d96c5-acba-413c-a429-8c9eaa67fe0d None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:28.753102 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4d96c5-acba-413c-a429-8c9eaa67fe0d None admin] PATH_INFO: `/v3/services` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:28.754093 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4d96c5-acba-413c-a429-8c9eaa67fe0d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:28.754448 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4d96c5-acba-413c-a429-8c9eaa67fe0d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:28.755946 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4d96c5-acba-413c-a429-8c9eaa67fe0d None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:28.762092 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 477/952] 10.222.0.58 () {56 vars in 1122 bytes} [Tue May 17 19:11:28 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:28.767944 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4d96c5-acba-413c-a429-8c9eaa67fe0d None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:28.829367 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcfd18a-7879-4e55-8e75-36d4518ba845 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:28.869840 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1664254-5ecf-420e-963c-98cd08cfa9d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:28.870888 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1664254-5ecf-420e-963c-98cd08cfa9d2 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:28.871394 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1664254-5ecf-420e-963c-98cd08cfa9d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:28.871897 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1664254-5ecf-420e-963c-98cd08cfa9d2 None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:28.872907 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1664254-5ecf-420e-963c-98cd08cfa9d2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:28.873478 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1664254-5ecf-420e-963c-98cd08cfa9d2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:28.875286 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1664254-5ecf-420e-963c-98cd08cfa9d2 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:28.888511 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4d96c5-acba-413c-a429-8c9eaa67fe0d None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:28.898525 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 476/953] 10.222.0.58 () {56 vars in 1112 bytes} [Tue May 17 19:11:28 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:28.935990 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab0367f-3b41-4dfe-aed6-b30f94e9fdd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:28.937086 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab0367f-3b41-4dfe-aed6-b30f94e9fdd7 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:28.937616 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab0367f-3b41-4dfe-aed6-b30f94e9fdd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:28.938087 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab0367f-3b41-4dfe-aed6-b30f94e9fdd7 None admin] PATH_INFO: `/v3/services` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:28.939051 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab0367f-3b41-4dfe-aed6-b30f94e9fdd7 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:28.940119 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab0367f-3b41-4dfe-aed6-b30f94e9fdd7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:28.941750 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab0367f-3b41-4dfe-aed6-b30f94e9fdd7 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:28.952488 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 478/954] 10.222.0.58 () {56 vars in 1122 bytes} [Tue May 17 19:11:28 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:28.954658 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1664254-5ecf-420e-963c-98cd08cfa9d2 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:29.069584 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1664254-5ecf-420e-963c-98cd08cfa9d2 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:29.109321 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10ab235-b26e-467a-889a-c3d91e2bbc86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:29.109984 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10ab235-b26e-467a-889a-c3d91e2bbc86 None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:29.110261 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10ab235-b26e-467a-889a-c3d91e2bbc86 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:29.110535 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10ab235-b26e-467a-889a-c3d91e2bbc86 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:29.111305 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10ab235-b26e-467a-889a-c3d91e2bbc86 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:29.112719 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10ab235-b26e-467a-889a-c3d91e2bbc86 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:29.212354 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 477/955] 10.222.0.58 () {60 vars in 1144 bytes} [Tue May 17 19:11:28 2022] POST /identity/v3/endpoints => generated 349 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:11:29.238001 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b76b6e4-597c-4b94-b1d5-d386492404d1 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:29.238518 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b76b6e4-597c-4b94-b1d5-d386492404d1 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:29.238596 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b76b6e4-597c-4b94-b1d5-d386492404d1 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:29.240223 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 479/956] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:11:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:11:29.246216 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e495c12b-4418-4c96-a0f5-e5662a00ce95 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:29.246540 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e495c12b-4418-4c96-a0f5-e5662a00ce95 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:29.246827 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e495c12b-4418-4c96-a0f5-e5662a00ce95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:29.265699 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-e495c12b-4418-4c96-a0f5-e5662a00ce95 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:11:29.267237 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-e495c12b-4418-4c96-a0f5-e5662a00ce95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:11:29.355749 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 478/957] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:11:29 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 17 19:11:29.372654 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b76b6e4-597c-4b94-b1d5-d386492404d1 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:29.476158 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b76b6e4-597c-4b94-b1d5-d386492404d1 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:29.509835 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19df125d-e5ca-45ac-98ea-789ab683aa5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:29.510632 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19df125d-e5ca-45ac-98ea-789ab683aa5a None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:29.510910 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19df125d-e5ca-45ac-98ea-789ab683aa5a None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:29.511183 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19df125d-e5ca-45ac-98ea-789ab683aa5a None admin] PATH_INFO: `/v3/services/compute` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:29.511874 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19df125d-e5ca-45ac-98ea-789ab683aa5a None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:29.513243 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19df125d-e5ca-45ac-98ea-789ab683aa5a None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:29.525091 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-19df125d-e5ca-45ac-98ea-789ab683aa5a None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 17 19:11:29.525091 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:29.525091 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 17 19:11:29.525091 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 17 19:11:29.525091 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 17 19:11:29.525091 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 17 19:11:29.525091 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 17 19:11:29.525091 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 17 19:11:29.525091 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 17 19:11:29.525091 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:11:29.525091 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 17 19:11:29.525091 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:11:29.525091 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:29.525091 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:11:29.525091 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:11:29.525091 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:11:29.525091 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:11:29.526071 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:11:29.526071 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:11:29.526071 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:11:29.526071 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:11:29.526071 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:11:29.526071 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:11:29.526071 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 17 19:11:29.526071 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_service(service_id) May 17 19:11:29.526071 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 17 19:11:29.526071 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 17 19:11:29.526071 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:11:29.526071 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:11:29.526071 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:11:29.526071 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:11:29.526071 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:11:29.526071 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:11:29.526906 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:11:29.526906 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:11:29.526906 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:11:29.526906 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:11:29.526906 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:11:29.526906 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:11:29.526906 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:11:29.526906 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:11:29.526906 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:11:29.526906 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:11:29.526906 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 17 19:11:29.526906 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 17 19:11:29.526906 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 17 19:11:29.526906 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:11:29.531203 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 480/958] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 17 19:11:29 2022] GET /identity/v3/services/compute => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:11:29.532605 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e495c12b-4418-4c96-a0f5-e5662a00ce95 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:29.637935 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e495c12b-4418-4c96-a0f5-e5662a00ce95 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:29.672405 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5ea5ad-e05b-4ade-aa30-6b12c3bdf725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:29.673078 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5ea5ad-e05b-4ade-aa30-6b12c3bdf725 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:29.673360 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5ea5ad-e05b-4ade-aa30-6b12c3bdf725 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:29.673634 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5ea5ad-e05b-4ade-aa30-6b12c3bdf725 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:29.674449 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5ea5ad-e05b-4ade-aa30-6b12c3bdf725 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:29.674753 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5ea5ad-e05b-4ade-aa30-6b12c3bdf725 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:29.676228 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5ea5ad-e05b-4ade-aa30-6b12c3bdf725 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:29.685929 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 479/959] 10.222.0.58 () {56 vars in 1112 bytes} [Tue May 17 19:11:29 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:11:29.687264 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19df125d-e5ca-45ac-98ea-789ab683aa5a None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:29.787512 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c586cb0-cc0a-4900-b411-8abda1876620 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:29.787512 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c586cb0-cc0a-4900-b411-8abda1876620 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:29.787512 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c586cb0-cc0a-4900-b411-8abda1876620 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:29.787512 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 480/960] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:11:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 17 19:11:29.790260 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19df125d-e5ca-45ac-98ea-789ab683aa5a None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:29.790688 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb9a8ae-b77e-471b-a7b1-df9a827986b4 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:29.790972 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb9a8ae-b77e-471b-a7b1-df9a827986b4 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:29.791252 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb9a8ae-b77e-471b-a7b1-df9a827986b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:29.808377 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-1eb9a8ae-b77e-471b-a7b1-df9a827986b4 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:11:29.809557 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-1eb9a8ae-b77e-471b-a7b1-df9a827986b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:11:29.824540 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaad1cd-f9f6-4383-a28e-a5c8c5290883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:29.825202 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaad1cd-f9f6-4383-a28e-a5c8c5290883 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:29.825479 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaad1cd-f9f6-4383-a28e-a5c8c5290883 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:29.825746 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaad1cd-f9f6-4383-a28e-a5c8c5290883 None admin] PATH_INFO: `/v3/services` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:29.826454 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaad1cd-f9f6-4383-a28e-a5c8c5290883 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:29.826750 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaad1cd-f9f6-4383-a28e-a5c8c5290883 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:29.828132 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaad1cd-f9f6-4383-a28e-a5c8c5290883 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:29.833020 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 481/961] 10.222.0.58 () {56 vars in 1112 bytes} [Tue May 17 19:11:29 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:29.890597 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 481/962] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:11:29 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 17 19:11:29.914575 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaad1cd-f9f6-4383-a28e-a5c8c5290883 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:30.027106 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaad1cd-f9f6-4383-a28e-a5c8c5290883 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:30.063116 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0066a53-605f-4978-a63a-cafe5b7e8511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:30.063975 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0066a53-605f-4978-a63a-cafe5b7e8511 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:30.064253 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0066a53-605f-4978-a63a-cafe5b7e8511 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:30.064629 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0066a53-605f-4978-a63a-cafe5b7e8511 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:30.074477 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0066a53-605f-4978-a63a-cafe5b7e8511 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 17 19:11:30.083005 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-d0066a53-605f-4978-a63a-cafe5b7e8511 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 17 19:11:30.083005 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:30.083005 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:11:30.083005 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:11:30.083005 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:11:30.083005 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:11:30.083005 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:11:30.083005 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:11:30.083005 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:11:30.083005 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:11:30.083005 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:11:30.083005 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:11:30.083005 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 17 19:11:30.083005 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) May 17 19:11:30.083005 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 17 19:11:30.083005 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 17 19:11:30.083998 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 17 19:11:30.083998 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 17 19:11:30.083998 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 17 19:11:30.083998 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 17 19:11:30.083998 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:11:30.083998 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:11:30.083998 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:11:30.083998 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:11:30.083998 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:11:30.083998 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:11:30.083998 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:11:30.083998 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:11:30.083998 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:11:30.083998 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:11:30.083998 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:11:30.083998 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:11:30.085034 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:11:30.085034 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:11:30.085034 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:11:30.085034 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:11:30.085034 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 17 19:11:30.085034 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 17 19:11:30.085034 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 17 19:11:30.085034 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 17 19:11:30.085034 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 17 19:11:30.085034 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 17 19:11:30.085034 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 17 19:11:30.085034 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:11:30.085034 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 482/963] 10.222.0.58 () {56 vars in 1120 bytes} [Tue May 17 19:11:29 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:11:30.092024 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb9a8ae-b77e-471b-a7b1-df9a827986b4 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:30.197084 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb9a8ae-b77e-471b-a7b1-df9a827986b4 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:30.231906 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d852b072-df28-4323-b12a-5e83c01103f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:30.233334 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d852b072-df28-4323-b12a-5e83c01103f2 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:30.233813 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d852b072-df28-4323-b12a-5e83c01103f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:30.234307 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d852b072-df28-4323-b12a-5e83c01103f2 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:30.235243 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d852b072-df28-4323-b12a-5e83c01103f2 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:30.235780 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d852b072-df28-4323-b12a-5e83c01103f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:30.237915 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d852b072-df28-4323-b12a-5e83c01103f2 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:30.251030 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 482/964] 10.222.0.58 () {56 vars in 1115 bytes} [Tue May 17 19:11:30 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:11:30.252839 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0066a53-605f-4978-a63a-cafe5b7e8511 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:30.354388 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0066a53-605f-4978-a63a-cafe5b7e8511 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:30.392056 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234d8879-a1b6-4f92-991b-be70ec5b0b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:30.393825 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234d8879-a1b6-4f92-991b-be70ec5b0b14 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:30.394328 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234d8879-a1b6-4f92-991b-be70ec5b0b14 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:30.394871 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234d8879-a1b6-4f92-991b-be70ec5b0b14 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:30.406224 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234d8879-a1b6-4f92-991b-be70ec5b0b14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:30.407939 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234d8879-a1b6-4f92-991b-be70ec5b0b14 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:30.424886 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-234d8879-a1b6-4f92-991b-be70ec5b0b14 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:11:30.424886 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:30.424886 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:11:30.424886 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:11:30.424886 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:11:30.424886 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:11:30.424886 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:11:30.424886 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:11:30.424886 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:11:30.424886 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:11:30.424886 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:11:30.424886 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:11:30.424886 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 17 19:11:30.424886 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 17 19:11:30.424886 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 17 19:11:30.425799 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 17 19:11:30.425799 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:11:30.425799 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:11:30.425799 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:11:30.425799 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:11:30.425799 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:11:30.425799 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:11:30.425799 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:11:30.425799 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:11:30.425799 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:11:30.425799 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:11:30.425799 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:11:30.425799 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:11:30.425799 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:11:30.425799 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:11:30.425799 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:11:30.425799 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:11:30.426687 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 17 19:11:30.426687 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 17 19:11:30.426687 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:11:30.426687 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:11:30.433241 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 483/965] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:11:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:11:30.434821 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d852b072-df28-4323-b12a-5e83c01103f2 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:30.535463 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d852b072-df28-4323-b12a-5e83c01103f2 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:30.570290 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c20649-b2fb-4d0d-92d3-50f695279a3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:30.571324 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c20649-b2fb-4d0d-92d3-50f695279a3e None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:30.571868 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c20649-b2fb-4d0d-92d3-50f695279a3e None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:30.572335 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c20649-b2fb-4d0d-92d3-50f695279a3e None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:30.573250 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c20649-b2fb-4d0d-92d3-50f695279a3e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:30.573766 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c20649-b2fb-4d0d-92d3-50f695279a3e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:30.575340 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c20649-b2fb-4d0d-92d3-50f695279a3e None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:30.588033 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 483/966] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 17 19:11:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:30.599615 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234d8879-a1b6-4f92-991b-be70ec5b0b14 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:30.702026 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234d8879-a1b6-4f92-991b-be70ec5b0b14 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:30.736083 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24cdc7f-a441-45ae-b3bc-3d952554880f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:30.737403 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24cdc7f-a441-45ae-b3bc-3d952554880f None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:30.737677 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24cdc7f-a441-45ae-b3bc-3d952554880f None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:30.737943 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24cdc7f-a441-45ae-b3bc-3d952554880f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:30.745764 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24cdc7f-a441-45ae-b3bc-3d952554880f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:30.747204 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24cdc7f-a441-45ae-b3bc-3d952554880f None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:30.761427 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-f24cdc7f-a441-45ae-b3bc-3d952554880f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 17 19:11:30.761427 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:30.761427 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:11:30.761427 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:11:30.761427 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:11:30.761427 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:11:30.761427 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:11:30.761427 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:11:30.761427 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:11:30.761427 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:11:30.761427 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:11:30.761427 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:11:30.761427 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 17 19:11:30.761427 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_user(user_id) May 17 19:11:30.761427 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 17 19:11:30.761427 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 17 19:11:30.762339 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:11:30.762339 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:11:30.762339 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 17 19:11:30.762339 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 17 19:11:30.762339 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 17 19:11:30.762339 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 17 19:11:30.762339 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:11:30.762339 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:11:30.762339 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:11:30.762339 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:11:30.762339 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:11:30.762339 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:11:30.762339 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:11:30.762339 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:11:30.762339 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:11:30.762339 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:11:30.762339 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:11:30.762339 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:11:30.763275 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:11:30.763275 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:11:30.763275 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 17 19:11:30.763275 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 17 19:11:30.763275 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 17 19:11:30.763275 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 17 19:11:30.763275 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 17 19:11:30.763275 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 17 19:11:30.763275 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 17 19:11:30.763275 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:11:30.763275 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 484/967] 10.222.0.58 () {56 vars in 1117 bytes} [Tue May 17 19:11:30 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:11:30.770496 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c20649-b2fb-4d0d-92d3-50f695279a3e None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:30.869559 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c20649-b2fb-4d0d-92d3-50f695279a3e None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:30.903418 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d24836-aa51-425e-bd12-1cc8acb44716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:30.904476 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d24836-aa51-425e-bd12-1cc8acb44716 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:30.904956 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d24836-aa51-425e-bd12-1cc8acb44716 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:30.905428 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d24836-aa51-425e-bd12-1cc8acb44716 None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:30.906466 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d24836-aa51-425e-bd12-1cc8acb44716 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:30.907011 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d24836-aa51-425e-bd12-1cc8acb44716 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:30.908689 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d24836-aa51-425e-bd12-1cc8acb44716 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:30.927523 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 484/968] 10.222.0.58 () {56 vars in 1149 bytes} [Tue May 17 19:11:30 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:30.933706 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24cdc7f-a441-45ae-b3bc-3d952554880f None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:31.041905 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24cdc7f-a441-45ae-b3bc-3d952554880f None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:31.075603 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a74e820-4073-409f-bde7-3a06adc201fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:31.076247 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a74e820-4073-409f-bde7-3a06adc201fa None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:31.076522 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a74e820-4073-409f-bde7-3a06adc201fa None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:31.076788 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a74e820-4073-409f-bde7-3a06adc201fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:31.151076 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a74e820-4073-409f-bde7-3a06adc201fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:31.152467 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a74e820-4073-409f-bde7-3a06adc201fa None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:31.252535 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 485/969] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 17 19:11:30 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 17 19:11:31.257750 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d24836-aa51-425e-bd12-1cc8acb44716 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:31.292096 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdf61c2-cf16-4ad9-9629-19a66223d0ba None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:31.292629 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdf61c2-cf16-4ad9-9629-19a66223d0ba None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:31.292629 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdf61c2-cf16-4ad9-9629-19a66223d0ba None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:31.294291 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 486/970] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:11:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 17 19:11:31.299634 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0761b2bc-cf82-452d-8f95-40ed7f32d90a None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:31.299914 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0761b2bc-cf82-452d-8f95-40ed7f32d90a None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:31.300184 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0761b2bc-cf82-452d-8f95-40ed7f32d90a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:31.317034 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-0761b2bc-cf82-452d-8f95-40ed7f32d90a None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:11:31.318187 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-0761b2bc-cf82-452d-8f95-40ed7f32d90a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:11:31.359068 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d24836-aa51-425e-bd12-1cc8acb44716 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:31.386615 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 487/971] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:11:31 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 17 19:11:31.398401 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5538df-eef7-4490-940b-a5f9e2d6cf8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:31.399270 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5538df-eef7-4490-940b-a5f9e2d6cf8e None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:31.399628 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5538df-eef7-4490-940b-a5f9e2d6cf8e None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:31.399902 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5538df-eef7-4490-940b-a5f9e2d6cf8e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:31.411758 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5538df-eef7-4490-940b-a5f9e2d6cf8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:31.412656 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0761b2bc-cf82-452d-8f95-40ed7f32d90a None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:31.416971 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5538df-eef7-4490-940b-a5f9e2d6cf8e None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:31.435570 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-eb5538df-eef7-4490-940b-a5f9e2d6cf8e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:11:31.435570 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:31.435570 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:11:31.435570 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:11:31.435570 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:11:31.435570 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:11:31.435570 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:11:31.435570 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:11:31.435570 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:11:31.435570 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:11:31.435570 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:11:31.435570 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:11:31.435570 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 17 19:11:31.435570 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 17 19:11:31.435570 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 17 19:11:31.436569 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 17 19:11:31.436569 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:11:31.436569 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:11:31.436569 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:11:31.436569 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:11:31.436569 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:11:31.436569 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:11:31.436569 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:11:31.436569 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:11:31.436569 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:11:31.436569 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:11:31.436569 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:11:31.436569 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:11:31.436569 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:11:31.436569 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:11:31.436569 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:11:31.436569 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:11:31.437531 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 17 19:11:31.437531 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 17 19:11:31.437531 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:11:31.437531 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:11:31.444329 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 485/972] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:11:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:11:31.445416 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5538df-eef7-4490-940b-a5f9e2d6cf8e None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:31.531389 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0761b2bc-cf82-452d-8f95-40ed7f32d90a None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:31.553532 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5538df-eef7-4490-940b-a5f9e2d6cf8e None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:31.569221 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c704b28c-f278-4382-a687-1264a95d0987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:31.570036 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c704b28c-f278-4382-a687-1264a95d0987 None admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:31.570319 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c704b28c-f278-4382-a687-1264a95d0987 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:31.570588 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c704b28c-f278-4382-a687-1264a95d0987 None admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:31.571297 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c704b28c-f278-4382-a687-1264a95d0987 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:31.572805 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c704b28c-f278-4382-a687-1264a95d0987 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:31.574285 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: WARNING keystone.server.flask.common [None req-c704b28c-f278-4382-a687-1264a95d0987 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 17 19:11:31.593112 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2048826f-7c68-485a-848c-543b434cb746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:31.593769 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2048826f-7c68-485a-848c-543b434cb746 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:31.594045 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2048826f-7c68-485a-848c-543b434cb746 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:31.594315 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2048826f-7c68-485a-848c-543b434cb746 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:31.601312 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2048826f-7c68-485a-848c-543b434cb746 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:31.601312 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2048826f-7c68-485a-848c-543b434cb746 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:31.601312 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2048826f-7c68-485a-848c-543b434cb746 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:31.608169 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 486/973] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 17 19:11:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:31.609482 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea86b2cf-7b3e-4c2b-9dfa-8a451371b8d0 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:31.609769 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea86b2cf-7b3e-4c2b-9dfa-8a451371b8d0 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:31.610045 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea86b2cf-7b3e-4c2b-9dfa-8a451371b8d0 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:31.611459 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 487/974] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:11:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:11:31.613464 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea86b2cf-7b3e-4c2b-9dfa-8a451371b8d0 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:31.624020 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.sql.core [None req-c704b28c-f278-4382-a687-1264a95d0987 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 17 19:11:31.624020 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 17 19:11:31.624020 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [parameters: {'user_id': '460d80439d3e4e569e3b22f725aca03a', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 17 19:11:31.624020 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80161) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 17 19:11:31.631436 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-c704b28c-f278-4382-a687-1264a95d0987 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 17 19:11:31.631436 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:31.631436 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 17 19:11:31.631436 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self.dialect.do_execute( May 17 19:11:31.631436 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 17 19:11:31.631436 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 17 19:11:31.631436 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 17 19:11:31.631436 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application result = self._query(query) May 17 19:11:31.631436 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 17 19:11:31.631436 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application conn.query(q) May 17 19:11:31.631436 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 17 19:11:31.631436 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 17 19:11:31.631436 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 17 19:11:31.631436 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application result.read() May 17 19:11:31.631436 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 17 19:11:31.634285 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 17 19:11:31.634285 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 17 19:11:31.634285 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application packet.raise_for_error() May 17 19:11:31.634285 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 17 19:11:31.634285 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 17 19:11:31.634285 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 17 19:11:31.634285 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 17 19:11:31.634285 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 17 19:11:31.634285 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:11:31.634285 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 17 19:11:31.634285 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:11:31.634285 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:31.634285 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 17 19:11:31.634285 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 17 19:11:31.634285 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user May 17 19:11:31.634285 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) May 17 19:11:31.634285 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 17 19:11:31.634285 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application next(self.gen) May 17 19:11:31.635262 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 17 19:11:31.635262 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application yield resource May 17 19:11:31.635262 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 17 19:11:31.635262 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application next(self.gen) May 17 19:11:31.635262 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 17 19:11:31.635262 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self.session.rollback() May 17 19:11:31.635262 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 17 19:11:31.635262 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self.force_reraise() May 17 19:11:31.635262 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 17 19:11:31.635262 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise self.value May 17 19:11:31.635262 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 17 19:11:31.635262 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 17 19:11:31.635262 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 17 19:11:31.635262 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application session.commit() May 17 19:11:31.635262 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 17 19:11:31.635262 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 17 19:11:31.635262 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 17 19:11:31.636212 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self._prepare_impl() May 17 19:11:31.636212 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 17 19:11:31.636212 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self.session.flush() May 17 19:11:31.636212 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 17 19:11:31.636212 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self._flush(objects) May 17 19:11:31.636212 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 17 19:11:31.636212 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 17 19:11:31.636212 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 17 19:11:31.636212 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application compat.raise_( May 17 19:11:31.636212 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 17 19:11:31.636212 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception May 17 19:11:31.636212 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 17 19:11:31.636212 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application flush_context.execute() May 17 19:11:31.636212 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 17 19:11:31.636212 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rec.execute(self) May 17 19:11:31.636212 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 17 19:11:31.636212 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 17 19:11:31.637207 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 17 19:11:31.637207 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application _emit_insert_statements( May 17 19:11:31.637207 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements May 17 19:11:31.637207 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application result = connection._execute_20( May 17 19:11:31.637207 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 17 19:11:31.637207 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 17 19:11:31.637207 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 17 19:11:31.637207 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 17 19:11:31.637207 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 17 19:11:31.637207 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ret = self._execute_context( May 17 19:11:31.637207 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 17 19:11:31.637207 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 17 19:11:31.637207 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 17 19:11:31.637207 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 17 19:11:31.637207 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 17 19:11:31.637207 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception May 17 19:11:31.639093 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 17 19:11:31.639093 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self.dialect.do_execute( May 17 19:11:31.639093 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 17 19:11:31.639093 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 17 19:11:31.639093 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 17 19:11:31.639093 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application result = self._query(query) May 17 19:11:31.639093 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 17 19:11:31.639093 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application conn.query(q) May 17 19:11:31.639093 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 17 19:11:31.639093 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 17 19:11:31.639093 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 17 19:11:31.639093 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application result.read() May 17 19:11:31.639093 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 17 19:11:31.639093 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 17 19:11:31.639093 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 17 19:11:31.639093 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application packet.raise_for_error() May 17 19:11:31.642201 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 17 19:11:31.642201 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 17 19:11:31.642201 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 17 19:11:31.642201 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 17 19:11:31.642201 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 17 19:11:31.642201 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 17 19:11:31.642201 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [parameters: {'user_id': '460d80439d3e4e569e3b22f725aca03a', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 17 19:11:31.642201 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 17 19:11:31.642201 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:11:31.642201 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 17 19:11:31.642201 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:11:31.642201 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:31.642201 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:11:31.642201 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:11:31.642201 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:11:31.645516 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:11:31.645516 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:11:31.645516 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:11:31.645516 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:11:31.645516 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:11:31.645516 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:11:31.645516 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:11:31.645516 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post May 17 19:11:31.645516 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( May 17 19:11:31.645516 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:11:31.645516 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:11:31.645516 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 17 19:11:31.645516 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 17 19:11:31.645516 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 17 19:11:31.645516 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 17 19:11:31.645516 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user May 17 19:11:31.645516 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) May 17 19:11:31.647567 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects May 17 19:11:31.647567 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) May 17 19:11:31.647567 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 17 19:11:31.647567 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 17 19:11:31.647567 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 17 19:11:31.647567 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:11:31.647567 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 488/975] 10.222.0.58 () {60 vars in 1131 bytes} [Tue May 17 19:11:31 2022] POST /identity/v3/users => generated 160 bytes in 221 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) May 17 19:11:31.647567 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11166ff-a18a-4b28-82b8-1ad03143e65a None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:31.647567 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11166ff-a18a-4b28-82b8-1ad03143e65a None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:31.647567 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11166ff-a18a-4b28-82b8-1ad03143e65a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:31.656067 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-e11166ff-a18a-4b28-82b8-1ad03143e65a None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:11:31.657571 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-e11166ff-a18a-4b28-82b8-1ad03143e65a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:11:31.721588 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea86b2cf-7b3e-4c2b-9dfa-8a451371b8d0 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:31.723524 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 489/976] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:11:31 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 17 19:11:31.723765 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11166ff-a18a-4b28-82b8-1ad03143e65a None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:31.762609 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f863c128-df75-4c99-8080-4d311555a421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:31.763739 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f863c128-df75-4c99-8080-4d311555a421 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:31.764146 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f863c128-df75-4c99-8080-4d311555a421 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:31.764420 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f863c128-df75-4c99-8080-4d311555a421 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:31.771349 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f863c128-df75-4c99-8080-4d311555a421 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:31.772830 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f863c128-df75-4c99-8080-4d311555a421 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:31.787431 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-f863c128-df75-4c99-8080-4d311555a421 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 17 19:11:31.787431 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:31.787431 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:11:31.787431 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:11:31.787431 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:11:31.787431 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:11:31.787431 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:11:31.787431 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:11:31.787431 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:11:31.787431 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:11:31.787431 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:11:31.787431 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:11:31.787431 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 17 19:11:31.787431 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(project_id) May 17 19:11:31.787431 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 17 19:11:31.788554 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 17 19:11:31.788554 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:11:31.788554 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:11:31.788554 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:11:31.788554 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:11:31.788554 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:11:31.788554 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:11:31.788554 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:11:31.788554 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:11:31.788554 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:11:31.788554 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:11:31.788554 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:11:31.788554 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:11:31.788554 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:11:31.788554 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:11:31.788554 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:11:31.788554 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:11:31.794358 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 17 19:11:31.794358 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 17 19:11:31.794358 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 17 19:11:31.794358 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 17 19:11:31.794358 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 17 19:11:31.794358 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 17 19:11:31.794358 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 17 19:11:31.794358 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:11:31.794358 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 488/977] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 17 19:11:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:11:31.794358 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f863c128-df75-4c99-8080-4d311555a421 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:31.835201 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11166ff-a18a-4b28-82b8-1ad03143e65a None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:31.873589 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228c991e-194d-4057-8023-29a5ccf970a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:31.874601 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228c991e-194d-4057-8023-29a5ccf970a1 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:31.874941 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228c991e-194d-4057-8023-29a5ccf970a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:31.875292 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228c991e-194d-4057-8023-29a5ccf970a1 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:31.882824 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228c991e-194d-4057-8023-29a5ccf970a1 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:31.884545 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228c991e-194d-4057-8023-29a5ccf970a1 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:31.886920 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f863c128-df75-4c99-8080-4d311555a421 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:31.898783 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-228c991e-194d-4057-8023-29a5ccf970a1 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 17 19:11:31.898783 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:31.898783 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:11:31.898783 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:11:31.898783 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:11:31.898783 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:11:31.898783 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:11:31.898783 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:11:31.898783 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:11:31.898783 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:11:31.898783 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:11:31.898783 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:11:31.898783 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 17 19:11:31.898783 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_user(user_id) May 17 19:11:31.898783 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 17 19:11:31.898783 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 17 19:11:31.900004 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:11:31.900004 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:11:31.900004 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 17 19:11:31.900004 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 17 19:11:31.900004 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 17 19:11:31.900004 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 17 19:11:31.900004 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:11:31.900004 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:11:31.900004 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:11:31.900004 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:11:31.900004 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:11:31.900004 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:11:31.900004 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:11:31.900004 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:11:31.900004 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:11:31.900004 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:11:31.900004 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:11:31.900004 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:11:31.901197 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:11:31.901197 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:11:31.901197 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 17 19:11:31.901197 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 17 19:11:31.901197 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 17 19:11:31.901197 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 17 19:11:31.901197 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 17 19:11:31.901197 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 17 19:11:31.901197 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 17 19:11:31.901197 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:11:31.902077 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 490/978] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 17 19:11:31 2022] GET /identity/v3/users/cinder => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:11:31.902853 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228c991e-194d-4057-8023-29a5ccf970a1 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:31.924999 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47901e7e-7a82-4ad7-983e-242ba87b8ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:31.925805 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47901e7e-7a82-4ad7-983e-242ba87b8ffc None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:31.926085 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47901e7e-7a82-4ad7-983e-242ba87b8ffc None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:31.926356 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47901e7e-7a82-4ad7-983e-242ba87b8ffc None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:31.926997 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47901e7e-7a82-4ad7-983e-242ba87b8ffc None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:31.928387 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47901e7e-7a82-4ad7-983e-242ba87b8ffc None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:31.942043 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-47901e7e-7a82-4ad7-983e-242ba87b8ffc None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 17 19:11:31.942043 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:31.942043 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 17 19:11:31.942043 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 17 19:11:31.942043 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 17 19:11:31.942043 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 17 19:11:31.942043 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 17 19:11:31.942043 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 17 19:11:31.942043 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 17 19:11:31.942043 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:11:31.942043 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 17 19:11:31.942043 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:11:31.942043 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:31.942043 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:11:31.942043 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:11:31.942043 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:11:31.942043 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:11:31.943093 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:11:31.943093 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:11:31.943093 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:11:31.943093 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:11:31.943093 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:11:31.943093 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:11:31.943093 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 17 19:11:31.943093 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(service_id) May 17 19:11:31.943093 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 17 19:11:31.943093 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 17 19:11:31.943093 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:11:31.943093 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:11:31.943093 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:11:31.943093 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:11:31.943093 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:11:31.943093 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:11:31.943971 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:11:31.943971 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:11:31.943971 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:11:31.943971 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:11:31.943971 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:11:31.943971 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:11:31.943971 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:11:31.943971 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:11:31.943971 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:11:31.943971 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:11:31.943971 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 17 19:11:31.943971 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 17 19:11:31.943971 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 17 19:11:31.943971 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:11:31.943971 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 489/979] 10.222.0.58 () {56 vars in 1126 bytes} [Tue May 17 19:11:31 2022] GET /identity/v3/services/object-store => generated 93 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:11:31.944877 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47901e7e-7a82-4ad7-983e-242ba87b8ffc None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:32.012051 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228c991e-194d-4057-8023-29a5ccf970a1 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:32.046594 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1ceff5-f5aa-41fb-b9a3-1a930629a5e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:32.047248 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1ceff5-f5aa-41fb-b9a3-1a930629a5e5 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:32.047559 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1ceff5-f5aa-41fb-b9a3-1a930629a5e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:32.047804 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47901e7e-7a82-4ad7-983e-242ba87b8ffc None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:32.047907 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1ceff5-f5aa-41fb-b9a3-1a930629a5e5 None admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:32.049100 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1ceff5-f5aa-41fb-b9a3-1a930629a5e5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:32.049415 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1ceff5-f5aa-41fb-b9a3-1a930629a5e5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:32.050866 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1ceff5-f5aa-41fb-b9a3-1a930629a5e5 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:32.070284 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 491/980] 10.222.0.58 () {56 vars in 1148 bytes} [Tue May 17 19:11:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:32.070284 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1ceff5-f5aa-41fb-b9a3-1a930629a5e5 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:32.082574 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a20646f-ef05-477b-b552-cfbb31c5b181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:32.083245 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a20646f-ef05-477b-b552-cfbb31c5b181 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:32.083547 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a20646f-ef05-477b-b552-cfbb31c5b181 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:32.083877 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a20646f-ef05-477b-b552-cfbb31c5b181 None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:32.084677 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a20646f-ef05-477b-b552-cfbb31c5b181 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:32.085038 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a20646f-ef05-477b-b552-cfbb31c5b181 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:32.086492 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a20646f-ef05-477b-b552-cfbb31c5b181 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:32.121550 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 490/981] 10.222.0.58 () {56 vars in 1101 bytes} [Tue May 17 19:11:31 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:32.128588 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a20646f-ef05-477b-b552-cfbb31c5b181 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:32.182600 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1ceff5-f5aa-41fb-b9a3-1a930629a5e5 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:32.219853 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a655aa-ddbc-4e7d-b688-7b2a8d191136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:32.220563 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a655aa-ddbc-4e7d-b688-7b2a8d191136 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:32.221091 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a655aa-ddbc-4e7d-b688-7b2a8d191136 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:32.221357 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a655aa-ddbc-4e7d-b688-7b2a8d191136 None admin] PATH_INFO: `/v3/services` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:32.222086 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a655aa-ddbc-4e7d-b688-7b2a8d191136 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:32.222383 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a655aa-ddbc-4e7d-b688-7b2a8d191136 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:32.223790 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a655aa-ddbc-4e7d-b688-7b2a8d191136 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:32.228962 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 492/982] 10.222.0.58 () {56 vars in 1122 bytes} [Tue May 17 19:11:32 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:32.228962 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7807fd96-8c48-4708-b4d8-0a9f2533f1dc None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:32.228962 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7807fd96-8c48-4708-b4d8-0a9f2533f1dc None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:32.228962 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7807fd96-8c48-4708-b4d8-0a9f2533f1dc None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:32.228962 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 493/983] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:11:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:11:32.237567 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7807fd96-8c48-4708-b4d8-0a9f2533f1dc None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:32.244554 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a20646f-ef05-477b-b552-cfbb31c5b181 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:32.280250 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56af8b19-524e-4ddf-ad72-9401fb5794d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:32.280914 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56af8b19-524e-4ddf-ad72-9401fb5794d5 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:32.281203 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56af8b19-524e-4ddf-ad72-9401fb5794d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:32.281480 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56af8b19-524e-4ddf-ad72-9401fb5794d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:32.282323 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56af8b19-524e-4ddf-ad72-9401fb5794d5 None admin] RBAC: Adding query filter params (role.id=897862af1c83498f86d0dfdf48bea276, scope.project.id=1a9d3a0631284c9fb113266036993e2d, user.id=2b90b57c542440448579b3aadda8b241) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:32.282635 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56af8b19-524e-4ddf-ad72-9401fb5794d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:32.284203 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56af8b19-524e-4ddf-ad72-9401fb5794d5 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:32.295524 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 491/984] 10.222.0.58 () {56 vars in 1374 bytes} [Tue May 17 19:11:32 2022] GET /identity/v3/role_assignments?user.id=2b90b57c542440448579b3aadda8b241&scope.project.id=1a9d3a0631284c9fb113266036993e2d&role.id=897862af1c83498f86d0dfdf48bea276 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:32.295524 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a5c12b-09d6-4056-9e68-659185c1801b None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:32.295524 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a5c12b-09d6-4056-9e68-659185c1801b None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:32.295524 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a5c12b-09d6-4056-9e68-659185c1801b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:32.312503 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-56a5c12b-09d6-4056-9e68-659185c1801b None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:11:32.315245 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-56a5c12b-09d6-4056-9e68-659185c1801b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:11:32.350921 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7807fd96-8c48-4708-b4d8-0a9f2533f1dc None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:32.390958 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 492/985] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:11:32 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 17 19:11:32.391317 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be84df39-e1c7-46ab-9200-32cfb8f4f3a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:32.392044 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be84df39-e1c7-46ab-9200-32cfb8f4f3a7 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:32.392377 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be84df39-e1c7-46ab-9200-32cfb8f4f3a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:32.392630 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be84df39-e1c7-46ab-9200-32cfb8f4f3a7 None admin] PATH_INFO: `/v3/services` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:32.393346 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be84df39-e1c7-46ab-9200-32cfb8f4f3a7 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:32.393644 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be84df39-e1c7-46ab-9200-32cfb8f4f3a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:32.395031 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be84df39-e1c7-46ab-9200-32cfb8f4f3a7 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:32.401791 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 494/986] 10.222.0.58 () {56 vars in 1122 bytes} [Tue May 17 19:11:32 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:32.407281 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a5c12b-09d6-4056-9e68-659185c1801b None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:32.417907 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be84df39-e1c7-46ab-9200-32cfb8f4f3a7 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:32.516446 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a5c12b-09d6-4056-9e68-659185c1801b None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:32.536498 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be84df39-e1c7-46ab-9200-32cfb8f4f3a7 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:32.554748 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82620835-e8a5-44fc-ba79-1d1f9732c064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:32.555399 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82620835-e8a5-44fc-ba79-1d1f9732c064 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:32.555719 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82620835-e8a5-44fc-ba79-1d1f9732c064 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:32.555988 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82620835-e8a5-44fc-ba79-1d1f9732c064 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:32.557020 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82620835-e8a5-44fc-ba79-1d1f9732c064 None admin] RBAC: Adding query filter params (interface=admin, service_id=948e60b6a47d46a6a5e964b5e3260b73, region_id=RegionOne) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:32.557152 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82620835-e8a5-44fc-ba79-1d1f9732c064 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:32.558413 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82620835-e8a5-44fc-ba79-1d1f9732c064 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:32.563232 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 493/987] 10.222.0.58 () {56 vars in 1249 bytes} [Tue May 17 19:11:32 2022] GET /identity/v3/endpoints?service_id=948e60b6a47d46a6a5e964b5e3260b73&interface=admin®ion_id=RegionOne => generated 193 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:32.568572 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82620835-e8a5-44fc-ba79-1d1f9732c064 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:32.570963 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed3c848-0708-4af9-b0bc-4c2e97682c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:32.571769 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed3c848-0708-4af9-b0bc-4c2e97682c51 None admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:32.572149 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed3c848-0708-4af9-b0bc-4c2e97682c51 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:32.572541 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed3c848-0708-4af9-b0bc-4c2e97682c51 None admin] PATH_INFO: `/v3/services` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:32.573408 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed3c848-0708-4af9-b0bc-4c2e97682c51 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:32.574925 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed3c848-0708-4af9-b0bc-4c2e97682c51 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:32.662251 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82620835-e8a5-44fc-ba79-1d1f9732c064 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:32.694456 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7022694-c620-4f22-a09e-732ff997b11d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:32.695104 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7022694-c620-4f22-a09e-732ff997b11d None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:32.695377 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7022694-c620-4f22-a09e-732ff997b11d None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:32.695672 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7022694-c620-4f22-a09e-732ff997b11d None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:32.696334 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7022694-c620-4f22-a09e-732ff997b11d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:32.698003 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7022694-c620-4f22-a09e-732ff997b11d None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:32.711642 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 494/988] 10.222.0.58 () {56 vars in 1087 bytes} [Tue May 17 19:11:32 2022] GET /identity/v3/services => generated 1727 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 17 19:11:32.711866 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 495/989] 10.222.0.58 () {60 vars in 1141 bytes} [Tue May 17 19:11:32 2022] POST /identity/v3/services => generated 242 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 17 19:11:34.564228 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272f9411-9243-42f3-8be7-d2fcc000617a None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:34.564755 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272f9411-9243-42f3-8be7-d2fcc000617a None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:34.564755 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272f9411-9243-42f3-8be7-d2fcc000617a None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:34.566314 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 495/990] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:11:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:11:34.571469 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd147126-1121-4fc5-95b2-0e4d4540d12d None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:34.571925 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd147126-1121-4fc5-95b2-0e4d4540d12d None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:34.572101 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd147126-1121-4fc5-95b2-0e4d4540d12d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:34.588889 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-fd147126-1121-4fc5-95b2-0e4d4540d12d None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:11:34.590073 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-fd147126-1121-4fc5-95b2-0e4d4540d12d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:11:34.664543 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 496/991] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:11:34 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 17 19:11:34.686810 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272f9411-9243-42f3-8be7-d2fcc000617a None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:34.792902 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272f9411-9243-42f3-8be7-d2fcc000617a None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:34.824893 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1a074a-4726-47dc-8f03-fdac274affcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:34.825593 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1a074a-4726-47dc-8f03-fdac274affcb None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:34.825868 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1a074a-4726-47dc-8f03-fdac274affcb None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:34.826135 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1a074a-4726-47dc-8f03-fdac274affcb None admin] PATH_INFO: `/v3/services/image` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:34.826774 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1a074a-4726-47dc-8f03-fdac274affcb None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:34.828188 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1a074a-4726-47dc-8f03-fdac274affcb None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:34.840038 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-9d1a074a-4726-47dc-8f03-fdac274affcb None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 17 19:11:34.840038 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:34.840038 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 17 19:11:34.840038 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 17 19:11:34.840038 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 17 19:11:34.840038 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 17 19:11:34.840038 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 17 19:11:34.840038 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 17 19:11:34.840038 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 17 19:11:34.840038 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:11:34.840038 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 17 19:11:34.840038 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:11:34.840038 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:34.840038 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:11:34.840038 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:11:34.840038 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:11:34.840038 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:11:34.841048 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:11:34.841048 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:11:34.841048 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:11:34.841048 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:11:34.841048 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:11:34.841048 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:11:34.841048 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 17 19:11:34.841048 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(service_id) May 17 19:11:34.841048 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 17 19:11:34.841048 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 17 19:11:34.841048 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:11:34.841048 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:11:34.841048 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:11:34.841048 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:11:34.841048 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:11:34.841048 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:11:34.848186 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:11:34.848186 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:11:34.848186 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:11:34.848186 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:11:34.848186 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:11:34.848186 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:11:34.848186 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:11:34.848186 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:11:34.848186 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:11:34.848186 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:11:34.848186 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 17 19:11:34.848186 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 17 19:11:34.848186 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 17 19:11:34.848186 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:11:34.848186 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 496/992] 10.222.0.58 () {56 vars in 1105 bytes} [Tue May 17 19:11:34 2022] GET /identity/v3/services/image => generated 86 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 17 19:11:34.848991 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd147126-1121-4fc5-95b2-0e4d4540d12d None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:34.926666 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b33f09-199b-400b-927f-56c7462990d0 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:34.926977 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b33f09-199b-400b-927f-56c7462990d0 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:34.927208 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b33f09-199b-400b-927f-56c7462990d0 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:34.928718 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 497/993] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:11:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 17 19:11:34.933314 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875b3004-332e-4ab9-b43a-e5c32154194a None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:34.933584 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875b3004-332e-4ab9-b43a-e5c32154194a None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:34.933850 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875b3004-332e-4ab9-b43a-e5c32154194a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:34.947889 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd147126-1121-4fc5-95b2-0e4d4540d12d None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:34.950048 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-875b3004-332e-4ab9-b43a-e5c32154194a None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:11:34.951249 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-875b3004-332e-4ab9-b43a-e5c32154194a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:11:34.980539 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547bb0e7-1439-4000-ac63-d642090ec6e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:34.981184 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547bb0e7-1439-4000-ac63-d642090ec6e0 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:34.981460 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547bb0e7-1439-4000-ac63-d642090ec6e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:34.981724 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547bb0e7-1439-4000-ac63-d642090ec6e0 None admin] PATH_INFO: `/v3/services` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:34.982416 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547bb0e7-1439-4000-ac63-d642090ec6e0 None admin] RBAC: Adding query filter params (name=image) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:34.982712 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547bb0e7-1439-4000-ac63-d642090ec6e0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:34.984090 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547bb0e7-1439-4000-ac63-d642090ec6e0 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:34.988658 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 497/994] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:11:34 2022] GET /identity/v3/services?name=image => generated 122 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:11:34.989821 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4196a92-3b51-40e7-ace8-f31e1c43cf66 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:34.990104 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4196a92-3b51-40e7-ace8-f31e1c43cf66 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:34.990367 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4196a92-3b51-40e7-ace8-f31e1c43cf66 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:34.991589 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 498/995] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:11:34 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:11:34.994088 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4196a92-3b51-40e7-ace8-f31e1c43cf66 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:35.016159 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 498/996] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:11:34 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 17 19:11:35.017936 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b65dafb-980b-4cfd-a71d-8ce2db457782 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:35.018220 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b65dafb-980b-4cfd-a71d-8ce2db457782 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:35.018490 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b65dafb-980b-4cfd-a71d-8ce2db457782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:35.034908 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-0b65dafb-980b-4cfd-a71d-8ce2db457782 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:11:35.036147 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-0b65dafb-980b-4cfd-a71d-8ce2db457782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:11:35.096606 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4196a92-3b51-40e7-ace8-f31e1c43cf66 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:35.099161 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 499/997] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:11:35 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 17 19:11:35.100026 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b65dafb-980b-4cfd-a71d-8ce2db457782 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:35.129557 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435a83ad-f4f8-4677-97c9-0c002f4ad95b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:35.130212 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435a83ad-f4f8-4677-97c9-0c002f4ad95b None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:35.130490 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435a83ad-f4f8-4677-97c9-0c002f4ad95b None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:35.130761 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435a83ad-f4f8-4677-97c9-0c002f4ad95b None admin] PATH_INFO: `/v3/services` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:35.131466 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435a83ad-f4f8-4677-97c9-0c002f4ad95b None admin] RBAC: Adding query filter params (type=image) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:35.131799 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435a83ad-f4f8-4677-97c9-0c002f4ad95b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:35.133164 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435a83ad-f4f8-4677-97c9-0c002f4ad95b None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:35.137408 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 499/998] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:11:34 2022] GET /identity/v3/services?type=image => generated 122 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:35.138166 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435a83ad-f4f8-4677-97c9-0c002f4ad95b None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:35.208421 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b65dafb-980b-4cfd-a71d-8ce2db457782 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:35.244256 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491036f7-86e2-4548-9de3-648f163fb287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:35.244744 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435a83ad-f4f8-4677-97c9-0c002f4ad95b None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:35.245065 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491036f7-86e2-4548-9de3-648f163fb287 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:35.245336 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491036f7-86e2-4548-9de3-648f163fb287 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:35.245605 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491036f7-86e2-4548-9de3-648f163fb287 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:35.246249 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491036f7-86e2-4548-9de3-648f163fb287 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:35.247713 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491036f7-86e2-4548-9de3-648f163fb287 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:35.260585 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-491036f7-86e2-4548-9de3-648f163fb287 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 17 19:11:35.260585 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:35.260585 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 17 19:11:35.260585 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 17 19:11:35.260585 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 17 19:11:35.260585 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 17 19:11:35.260585 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 17 19:11:35.260585 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 17 19:11:35.260585 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 17 19:11:35.260585 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:11:35.260585 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 17 19:11:35.260585 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:11:35.260585 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:35.260585 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:11:35.260585 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:11:35.260585 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:11:35.260585 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:11:35.261703 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:11:35.261703 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:11:35.261703 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:11:35.261703 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:11:35.261703 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:11:35.261703 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:11:35.261703 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 17 19:11:35.261703 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(service_id) May 17 19:11:35.261703 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 17 19:11:35.261703 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 17 19:11:35.261703 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:11:35.261703 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:11:35.261703 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:11:35.261703 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:11:35.261703 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:11:35.261703 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:11:35.262653 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:11:35.262653 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:11:35.262653 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:11:35.262653 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:11:35.262653 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:11:35.262653 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:11:35.262653 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:11:35.262653 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:11:35.262653 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:11:35.262653 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:11:35.262653 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 17 19:11:35.262653 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 17 19:11:35.262653 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 17 19:11:35.262653 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:11:35.262653 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 500/999] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 17 19:11:35 2022] GET /identity/v3/services/compute => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:11:35.268789 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491036f7-86e2-4548-9de3-648f163fb287 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:35.286514 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a83e730-73df-4120-9912-26f6e962d2da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:35.287506 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a83e730-73df-4120-9912-26f6e962d2da None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:35.287788 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a83e730-73df-4120-9912-26f6e962d2da None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:35.288382 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a83e730-73df-4120-9912-26f6e962d2da None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:35.289115 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a83e730-73df-4120-9912-26f6e962d2da None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:35.290490 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a83e730-73df-4120-9912-26f6e962d2da None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:35.306851 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-5a83e730-73df-4120-9912-26f6e962d2da None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 17 19:11:35.306851 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:35.306851 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 17 19:11:35.306851 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 17 19:11:35.306851 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 17 19:11:35.306851 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 17 19:11:35.306851 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 17 19:11:35.306851 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 17 19:11:35.306851 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 17 19:11:35.306851 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:11:35.306851 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 17 19:11:35.306851 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:11:35.306851 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:35.306851 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:11:35.306851 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:11:35.306851 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:11:35.306851 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:11:35.315012 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:11:35.315012 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:11:35.315012 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:11:35.315012 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:11:35.315012 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:11:35.315012 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:11:35.315012 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 17 19:11:35.315012 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_service(service_id) May 17 19:11:35.315012 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 17 19:11:35.315012 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 17 19:11:35.315012 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:11:35.315012 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:11:35.315012 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:11:35.315012 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:11:35.315012 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:11:35.315012 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:11:35.317246 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:11:35.317246 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:11:35.317246 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:11:35.317246 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:11:35.317246 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:11:35.317246 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:11:35.317246 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:11:35.317246 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:11:35.317246 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:11:35.317246 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:11:35.317246 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 17 19:11:35.317246 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 17 19:11:35.317246 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 17 19:11:35.317246 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:11:35.317246 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 500/1000] 10.222.0.58 () {56 vars in 1126 bytes} [Tue May 17 19:11:35 2022] GET /identity/v3/services/object-store => generated 93 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:11:35.321401 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a83e730-73df-4120-9912-26f6e962d2da None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:35.370592 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491036f7-86e2-4548-9de3-648f163fb287 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:35.403514 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19704c8f-81b8-42ac-be11-cddaef605218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:35.404177 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19704c8f-81b8-42ac-be11-cddaef605218 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:35.404451 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19704c8f-81b8-42ac-be11-cddaef605218 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:35.404721 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19704c8f-81b8-42ac-be11-cddaef605218 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:35.405413 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19704c8f-81b8-42ac-be11-cddaef605218 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:35.405711 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19704c8f-81b8-42ac-be11-cddaef605218 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:35.407071 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19704c8f-81b8-42ac-be11-cddaef605218 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:35.411939 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 501/1001] 10.222.0.58 () {56 vars in 1112 bytes} [Tue May 17 19:11:35 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:11:35.417298 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19704c8f-81b8-42ac-be11-cddaef605218 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:35.421110 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a83e730-73df-4120-9912-26f6e962d2da None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:35.453503 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b983fce-d682-418b-b125-366e6fa25c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:35.454155 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b983fce-d682-418b-b125-366e6fa25c7b None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:35.454429 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b983fce-d682-418b-b125-366e6fa25c7b None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:35.454751 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b983fce-d682-418b-b125-366e6fa25c7b None admin] PATH_INFO: `/v3/services` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:35.455955 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b983fce-d682-418b-b125-366e6fa25c7b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:35.456259 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b983fce-d682-418b-b125-366e6fa25c7b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:35.457614 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b983fce-d682-418b-b125-366e6fa25c7b None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:35.462589 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 501/1002] 10.222.0.58 () {56 vars in 1122 bytes} [Tue May 17 19:11:35 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:35.467880 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b983fce-d682-418b-b125-366e6fa25c7b None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:35.516139 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19704c8f-81b8-42ac-be11-cddaef605218 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:35.550766 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219b8be4-d46c-4f8f-8b35-f54329fc91e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:35.551434 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219b8be4-d46c-4f8f-8b35-f54329fc91e7 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:35.551743 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219b8be4-d46c-4f8f-8b35-f54329fc91e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:35.552433 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219b8be4-d46c-4f8f-8b35-f54329fc91e7 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:35.553140 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219b8be4-d46c-4f8f-8b35-f54329fc91e7 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:35.553446 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219b8be4-d46c-4f8f-8b35-f54329fc91e7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:35.554811 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219b8be4-d46c-4f8f-8b35-f54329fc91e7 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:35.559587 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 502/1003] 10.222.0.58 () {56 vars in 1112 bytes} [Tue May 17 19:11:35 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:11:35.563237 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b983fce-d682-418b-b125-366e6fa25c7b None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:35.565911 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219b8be4-d46c-4f8f-8b35-f54329fc91e7 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:35.596311 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca865b1-1ca5-46b3-8bc4-b412fdb56fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:35.596969 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca865b1-1ca5-46b3-8bc4-b412fdb56fbe None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:35.597234 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca865b1-1ca5-46b3-8bc4-b412fdb56fbe None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:35.597501 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca865b1-1ca5-46b3-8bc4-b412fdb56fbe None admin] PATH_INFO: `/v3/services` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:35.598298 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca865b1-1ca5-46b3-8bc4-b412fdb56fbe None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:35.598603 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca865b1-1ca5-46b3-8bc4-b412fdb56fbe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:35.599984 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca865b1-1ca5-46b3-8bc4-b412fdb56fbe None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:35.609332 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 502/1004] 10.222.0.58 () {56 vars in 1122 bytes} [Tue May 17 19:11:35 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:11:35.610176 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca865b1-1ca5-46b3-8bc4-b412fdb56fbe None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:35.664380 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219b8be4-d46c-4f8f-8b35-f54329fc91e7 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:35.697800 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e298c3-fd77-495d-9cb9-dde47c4d4761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:35.698458 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e298c3-fd77-495d-9cb9-dde47c4d4761 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:35.698738 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e298c3-fd77-495d-9cb9-dde47c4d4761 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:35.699011 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e298c3-fd77-495d-9cb9-dde47c4d4761 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:35.699829 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e298c3-fd77-495d-9cb9-dde47c4d4761 None admin] RBAC: Adding query filter params (interface=public, service_id=f6053bdcb5f84b1090a745bc20615cbc, region_id=RegionOne) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:35.700144 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e298c3-fd77-495d-9cb9-dde47c4d4761 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:35.701509 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e298c3-fd77-495d-9cb9-dde47c4d4761 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:35.711196 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 503/1005] 10.222.0.58 () {56 vars in 1251 bytes} [Tue May 17 19:11:35 2022] GET /identity/v3/endpoints?service_id=f6053bdcb5f84b1090a745bc20615cbc&interface=public®ion_id=RegionOne => generated 194 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:11:35.711824 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca865b1-1ca5-46b3-8bc4-b412fdb56fbe None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:35.713604 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e298c3-fd77-495d-9cb9-dde47c4d4761 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:35.744807 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ec8feb-6871-4713-b3f8-c0dff9ef0d70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:35.745456 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ec8feb-6871-4713-b3f8-c0dff9ef0d70 None admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:35.745729 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ec8feb-6871-4713-b3f8-c0dff9ef0d70 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:35.746000 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ec8feb-6871-4713-b3f8-c0dff9ef0d70 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:35.746725 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ec8feb-6871-4713-b3f8-c0dff9ef0d70 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:35.748093 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ec8feb-6871-4713-b3f8-c0dff9ef0d70 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:35.807523 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e298c3-fd77-495d-9cb9-dde47c4d4761 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:35.824696 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 503/1006] 10.222.0.58 () {60 vars in 1144 bytes} [Tue May 17 19:11:35 2022] POST /identity/v3/endpoints => generated 325 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:11:35.849287 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c391dc-21c1-46bc-bf73-ece81a590c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:35.849958 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c391dc-21c1-46bc-bf73-ece81a590c86 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:35.850233 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c391dc-21c1-46bc-bf73-ece81a590c86 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:35.850500 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c391dc-21c1-46bc-bf73-ece81a590c86 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:35.851177 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c391dc-21c1-46bc-bf73-ece81a590c86 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:35.852642 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c391dc-21c1-46bc-bf73-ece81a590c86 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:35.863968 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 504/1007] 10.222.0.58 () {56 vars in 1087 bytes} [Tue May 17 19:11:35 2022] GET /identity/v3/services => generated 1727 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:11:37.335838 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667ab6b5-83a6-4acc-9fec-a41e789b9b4a None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:37.336346 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667ab6b5-83a6-4acc-9fec-a41e789b9b4a None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:37.336346 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667ab6b5-83a6-4acc-9fec-a41e789b9b4a None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:37.337792 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 504/1008] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:11:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:11:37.342746 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de77f0f-88bd-4d20-8a7e-4416aa29ae8d None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:37.343158 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de77f0f-88bd-4d20-8a7e-4416aa29ae8d None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:37.343321 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de77f0f-88bd-4d20-8a7e-4416aa29ae8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:37.359988 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-4de77f0f-88bd-4d20-8a7e-4416aa29ae8d None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:11:37.361160 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-4de77f0f-88bd-4d20-8a7e-4416aa29ae8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:11:37.431478 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 505/1009] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:11:37 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 17 19:11:37.454508 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667ab6b5-83a6-4acc-9fec-a41e789b9b4a None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:37.553985 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667ab6b5-83a6-4acc-9fec-a41e789b9b4a None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:37.587102 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583e186f-9f9a-4f30-9cb9-70d8d5b2c42b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:37.587700 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583e186f-9f9a-4f30-9cb9-70d8d5b2c42b None admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:37.587980 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583e186f-9f9a-4f30-9cb9-70d8d5b2c42b None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:37.588248 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583e186f-9f9a-4f30-9cb9-70d8d5b2c42b None admin] PATH_INFO: `/v3/services` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:37.588965 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583e186f-9f9a-4f30-9cb9-70d8d5b2c42b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:37.590323 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583e186f-9f9a-4f30-9cb9-70d8d5b2c42b None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:37.699765 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 505/1010] 10.222.0.58 () {60 vars in 1141 bytes} [Tue May 17 19:11:37 2022] POST /identity/v3/services => generated 242 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:11:38.059404 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aaff41-81fd-4357-9db6-bcc1d75de0af None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:38.059808 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aaff41-81fd-4357-9db6-bcc1d75de0af None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:38.059997 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aaff41-81fd-4357-9db6-bcc1d75de0af None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:38.061501 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 506/1011] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:11:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:11:38.064517 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7345b5fa-7cd8-4fcd-b224-c67e2e24c055 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:38.064803 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7345b5fa-7cd8-4fcd-b224-c67e2e24c055 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:38.065076 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7345b5fa-7cd8-4fcd-b224-c67e2e24c055 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:38.066514 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 506/1012] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:11:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:11:38.066999 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc5ac08-b5d1-4259-b7e4-8f79c3b0ff3d None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:38.067288 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc5ac08-b5d1-4259-b7e4-8f79c3b0ff3d None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:38.067661 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc5ac08-b5d1-4259-b7e4-8f79c3b0ff3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:38.071231 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7db212c-5c06-42ca-92be-b1494ec299fb None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:38.071535 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7db212c-5c06-42ca-92be-b1494ec299fb None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:38.071812 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7db212c-5c06-42ca-92be-b1494ec299fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:38.085352 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-dfc5ac08-b5d1-4259-b7e4-8f79c3b0ff3d None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:11:38.086543 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-dfc5ac08-b5d1-4259-b7e4-8f79c3b0ff3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:11:38.088767 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-f7db212c-5c06-42ca-92be-b1494ec299fb None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:11:38.089907 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-f7db212c-5c06-42ca-92be-b1494ec299fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:11:38.162292 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 507/1013] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:11:38 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 17 19:11:38.162504 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 507/1014] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:11:38 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 17 19:11:38.181505 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc5ac08-b5d1-4259-b7e4-8f79c3b0ff3d None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:38.182727 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7db212c-5c06-42ca-92be-b1494ec299fb None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:38.282207 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7db212c-5c06-42ca-92be-b1494ec299fb None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:38.284725 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc5ac08-b5d1-4259-b7e4-8f79c3b0ff3d None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:38.315295 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab88747c-07b9-4e9e-9e5c-d38cc3572af5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:38.316261 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab88747c-07b9-4e9e-9e5c-d38cc3572af5 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:38.316571 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab88747c-07b9-4e9e-9e5c-d38cc3572af5 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:38.316798 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab88747c-07b9-4e9e-9e5c-d38cc3572af5 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:38.317436 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab88747c-07b9-4e9e-9e5c-d38cc3572af5 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:38.317828 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc5bba9-0050-4f06-8ca2-290d116390f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:38.318614 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc5bba9-0050-4f06-8ca2-290d116390f9 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:38.318795 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab88747c-07b9-4e9e-9e5c-d38cc3572af5 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:38.318872 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc5bba9-0050-4f06-8ca2-290d116390f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:38.319138 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc5bba9-0050-4f06-8ca2-290d116390f9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:38.320317 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc5bba9-0050-4f06-8ca2-290d116390f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:38.321804 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc5bba9-0050-4f06-8ca2-290d116390f9 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:38.323613 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 508/1015] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:11:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:38.329009 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc5bba9-0050-4f06-8ca2-290d116390f9 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:38.330836 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-ab88747c-07b9-4e9e-9e5c-d38cc3572af5 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 17 19:11:38.330836 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:38.330836 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 17 19:11:38.330836 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 17 19:11:38.330836 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 17 19:11:38.330836 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 17 19:11:38.330836 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 17 19:11:38.330836 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 17 19:11:38.330836 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 17 19:11:38.330836 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:11:38.330836 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 17 19:11:38.330836 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:11:38.330836 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:38.330836 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:11:38.330836 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:11:38.330836 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:11:38.330836 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:11:38.331826 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:11:38.331826 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:11:38.331826 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:11:38.331826 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:11:38.331826 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:11:38.331826 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:11:38.331826 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 17 19:11:38.331826 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_service(service_id) May 17 19:11:38.331826 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 17 19:11:38.331826 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 17 19:11:38.331826 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:11:38.331826 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:11:38.331826 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:11:38.331826 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:11:38.331826 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:11:38.331826 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:11:38.332658 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:11:38.332658 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:11:38.332658 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:11:38.332658 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:11:38.332658 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:11:38.332658 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:11:38.332658 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:11:38.332658 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:11:38.332658 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:11:38.332658 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:11:38.332658 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 17 19:11:38.332658 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 17 19:11:38.332658 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 17 19:11:38.332658 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:11:38.332658 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 508/1016] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 17 19:11:38 2022] GET /identity/v3/services/compute => generated 88 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 17 19:11:38.337806 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab88747c-07b9-4e9e-9e5c-d38cc3572af5 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:38.426033 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc5bba9-0050-4f06-8ca2-290d116390f9 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:38.433762 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab88747c-07b9-4e9e-9e5c-d38cc3572af5 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:38.461740 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f6085e-d36c-4bce-b64d-d6f848988284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:38.462400 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f6085e-d36c-4bce-b64d-d6f848988284 None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:38.462680 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f6085e-d36c-4bce-b64d-d6f848988284 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:38.462948 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f6085e-d36c-4bce-b64d-d6f848988284 None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:38.463684 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f6085e-d36c-4bce-b64d-d6f848988284 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:38.465104 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f6085e-d36c-4bce-b64d-d6f848988284 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:38.469418 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da332e3-9248-4bf1-8eac-0be9cbe81e65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:38.470179 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da332e3-9248-4bf1-8eac-0be9cbe81e65 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:38.470385 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da332e3-9248-4bf1-8eac-0be9cbe81e65 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:38.470655 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da332e3-9248-4bf1-8eac-0be9cbe81e65 None admin] PATH_INFO: `/v3/services` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:38.471651 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da332e3-9248-4bf1-8eac-0be9cbe81e65 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:38.471954 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da332e3-9248-4bf1-8eac-0be9cbe81e65 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:38.473300 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da332e3-9248-4bf1-8eac-0be9cbe81e65 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:38.478371 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 509/1017] 10.222.0.58 () {56 vars in 1112 bytes} [Tue May 17 19:11:38 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:11:38.483870 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da332e3-9248-4bf1-8eac-0be9cbe81e65 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:38.579561 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da332e3-9248-4bf1-8eac-0be9cbe81e65 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:38.590834 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 509/1018] 10.222.0.58 () {60 vars in 1141 bytes} [Tue May 17 19:11:38 2022] POST /identity/v3/projects => generated 311 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 17 19:11:38.614379 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4617cec3-e9bd-438b-9a9f-1584089b4a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:38.615336 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4617cec3-e9bd-438b-9a9f-1584089b4a92 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:38.615822 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4617cec3-e9bd-438b-9a9f-1584089b4a92 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:38.616239 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4617cec3-e9bd-438b-9a9f-1584089b4a92 None admin] PATH_INFO: `/v3/services` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:38.617199 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4617cec3-e9bd-438b-9a9f-1584089b4a92 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:38.617709 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4617cec3-e9bd-438b-9a9f-1584089b4a92 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:38.619320 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4617cec3-e9bd-438b-9a9f-1584089b4a92 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:38.625804 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 510/1019] 10.222.0.58 () {56 vars in 1112 bytes} [Tue May 17 19:11:38 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:38.632383 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f6085e-d36c-4bce-b64d-d6f848988284 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:38.737760 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f6085e-d36c-4bce-b64d-d6f848988284 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:38.772813 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d59387-a1fc-43bd-b3bd-1f0bdbac107f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:38.773473 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d59387-a1fc-43bd-b3bd-1f0bdbac107f None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:38.774405 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d59387-a1fc-43bd-b3bd-1f0bdbac107f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:38.774405 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d59387-a1fc-43bd-b3bd-1f0bdbac107f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:38.774775 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d59387-a1fc-43bd-b3bd-1f0bdbac107f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:38.776176 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d59387-a1fc-43bd-b3bd-1f0bdbac107f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:38.851243 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 510/1020] 10.222.0.58 () {60 vars in 1144 bytes} [Tue May 17 19:11:38 2022] POST /identity/v3/endpoints => generated 334 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:11:39.934633 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b711ac2-33de-45eb-ad2b-9752e80a9312 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:39.935186 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b711ac2-33de-45eb-ad2b-9752e80a9312 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:39.935186 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b711ac2-33de-45eb-ad2b-9752e80a9312 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:39.936601 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 511/1021] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:11:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:11:39.942188 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff39089-7ee1-47fd-b770-7f2724372521 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:39.942566 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff39089-7ee1-47fd-b770-7f2724372521 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:39.942739 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff39089-7ee1-47fd-b770-7f2724372521 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:39.959383 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-eff39089-7ee1-47fd-b770-7f2724372521 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:11:39.960570 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-eff39089-7ee1-47fd-b770-7f2724372521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:11:40.029098 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 511/1022] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:11:39 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 17 19:11:40.051817 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b711ac2-33de-45eb-ad2b-9752e80a9312 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:40.148298 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b711ac2-33de-45eb-ad2b-9752e80a9312 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:40.180732 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9619f92f-bedb-456c-949a-a742ac58e2b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:40.181844 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9619f92f-bedb-456c-949a-a742ac58e2b9 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:40.182318 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9619f92f-bedb-456c-949a-a742ac58e2b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:40.182814 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9619f92f-bedb-456c-949a-a742ac58e2b9 None admin] PATH_INFO: `/v3/services/image` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:40.183720 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9619f92f-bedb-456c-949a-a742ac58e2b9 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:40.185341 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9619f92f-bedb-456c-949a-a742ac58e2b9 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:40.197469 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-9619f92f-bedb-456c-949a-a742ac58e2b9 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 17 19:11:40.197469 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:40.197469 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 17 19:11:40.197469 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 17 19:11:40.197469 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 17 19:11:40.197469 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 17 19:11:40.197469 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 17 19:11:40.197469 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 17 19:11:40.197469 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 17 19:11:40.197469 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:11:40.197469 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 17 19:11:40.197469 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:11:40.197469 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:40.197469 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:11:40.197469 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:11:40.197469 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:11:40.197469 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:11:40.198459 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:11:40.198459 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:11:40.198459 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:11:40.198459 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:11:40.198459 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:11:40.198459 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:11:40.198459 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 17 19:11:40.198459 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_service(service_id) May 17 19:11:40.198459 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 17 19:11:40.198459 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 17 19:11:40.198459 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:11:40.198459 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:11:40.198459 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:11:40.198459 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:11:40.198459 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:11:40.198459 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:11:40.199292 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:11:40.199292 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:11:40.199292 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:11:40.199292 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:11:40.199292 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:11:40.199292 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:11:40.199292 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:11:40.199292 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:11:40.199292 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:11:40.199292 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:11:40.199292 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 17 19:11:40.199292 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 17 19:11:40.199292 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 17 19:11:40.199292 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:11:40.200955 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 512/1023] 10.222.0.58 () {56 vars in 1105 bytes} [Tue May 17 19:11:40 2022] GET /identity/v3/services/image => generated 86 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:11:40.206737 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff39089-7ee1-47fd-b770-7f2724372521 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:40.301169 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff39089-7ee1-47fd-b770-7f2724372521 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:40.333544 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ea9962-a889-4312-8cdf-22d9f96f9b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:40.334205 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea9962-a889-4312-8cdf-22d9f96f9b75 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:40.334482 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea9962-a889-4312-8cdf-22d9f96f9b75 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:40.334747 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea9962-a889-4312-8cdf-22d9f96f9b75 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:40.335436 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ea9962-a889-4312-8cdf-22d9f96f9b75 None admin] RBAC: Adding query filter params (name=image) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:40.335769 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ea9962-a889-4312-8cdf-22d9f96f9b75 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:40.337136 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ea9962-a889-4312-8cdf-22d9f96f9b75 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:40.347101 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 512/1024] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:11:40 2022] GET /identity/v3/services?name=image => generated 122 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:40.348393 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9619f92f-bedb-456c-949a-a742ac58e2b9 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:40.444195 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9619f92f-bedb-456c-949a-a742ac58e2b9 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:40.479473 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d03843e-4829-4909-84c3-f146abca56dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:40.480156 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d03843e-4829-4909-84c3-f146abca56dc None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:40.480430 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d03843e-4829-4909-84c3-f146abca56dc None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:40.480694 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d03843e-4829-4909-84c3-f146abca56dc None admin] PATH_INFO: `/v3/services` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:40.481387 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d03843e-4829-4909-84c3-f146abca56dc None admin] RBAC: Adding query filter params (type=image) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:40.481684 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d03843e-4829-4909-84c3-f146abca56dc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:40.483026 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d03843e-4829-4909-84c3-f146abca56dc None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:40.488305 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 513/1025] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:11:40 2022] GET /identity/v3/services?type=image => generated 350 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:40.493833 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ea9962-a889-4312-8cdf-22d9f96f9b75 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:40.594089 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ea9962-a889-4312-8cdf-22d9f96f9b75 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:40.627917 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339e44cb-ec54-4a26-b2f9-06147313120e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:40.628569 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339e44cb-ec54-4a26-b2f9-06147313120e None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:40.628845 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339e44cb-ec54-4a26-b2f9-06147313120e None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:40.629113 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339e44cb-ec54-4a26-b2f9-06147313120e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:40.629894 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339e44cb-ec54-4a26-b2f9-06147313120e None admin] RBAC: Adding query filter params (interface=public, service_id=be970b2f3d86458d88c028c9e7536657, region_id=RegionOne) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:40.630200 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339e44cb-ec54-4a26-b2f9-06147313120e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:40.631589 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339e44cb-ec54-4a26-b2f9-06147313120e None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:40.641759 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 513/1026] 10.222.0.58 () {56 vars in 1251 bytes} [Tue May 17 19:11:40 2022] GET /identity/v3/endpoints?service_id=be970b2f3d86458d88c028c9e7536657&interface=public®ion_id=RegionOne => generated 194 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:40.643726 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d03843e-4829-4909-84c3-f146abca56dc None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:40.743197 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d03843e-4829-4909-84c3-f146abca56dc None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:40.775240 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfb2aa0-a301-4a55-ae05-c4a471b5508f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:40.775930 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb2aa0-a301-4a55-ae05-c4a471b5508f None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:40.776208 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb2aa0-a301-4a55-ae05-c4a471b5508f None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:40.776486 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb2aa0-a301-4a55-ae05-c4a471b5508f None admin] PATH_INFO: `/v3/services` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:40.777161 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfb2aa0-a301-4a55-ae05-c4a471b5508f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:40.778513 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfb2aa0-a301-4a55-ae05-c4a471b5508f None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:40.791530 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 514/1027] 10.222.0.58 () {56 vars in 1087 bytes} [Tue May 17 19:11:40 2022] GET /identity/v3/services => generated 1957 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:11:40.834214 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c89919-3a32-4446-9fca-75ceaf26f60f None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:40.834497 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c89919-3a32-4446-9fca-75ceaf26f60f None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:40.834775 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c89919-3a32-4446-9fca-75ceaf26f60f None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:40.836197 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 514/1028] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:11:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:11:40.842001 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0131e95-0315-4a91-a77f-731335f32d85 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:40.842284 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0131e95-0315-4a91-a77f-731335f32d85 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:40.842562 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0131e95-0315-4a91-a77f-731335f32d85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:40.861668 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-b0131e95-0315-4a91-a77f-731335f32d85 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:11:40.862876 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-b0131e95-0315-4a91-a77f-731335f32d85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:11:40.932756 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 515/1029] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:11:40 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 17 19:11:40.958166 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c89919-3a32-4446-9fca-75ceaf26f60f None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:41.061200 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c89919-3a32-4446-9fca-75ceaf26f60f None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:41.093985 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a6b50f-57a2-43fb-8d3c-4dde13874ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:41.094773 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a6b50f-57a2-43fb-8d3c-4dde13874ea9 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:41.095599 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a6b50f-57a2-43fb-8d3c-4dde13874ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:41.096125 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a6b50f-57a2-43fb-8d3c-4dde13874ea9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:41.097285 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a6b50f-57a2-43fb-8d3c-4dde13874ea9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:41.098716 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a6b50f-57a2-43fb-8d3c-4dde13874ea9 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:41.100534 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 515/1030] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:11:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:41.106068 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0131e95-0315-4a91-a77f-731335f32d85 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:41.125487 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36084ac-0597-4d7c-a321-fc29efb662f9 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:41.125771 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36084ac-0597-4d7c-a321-fc29efb662f9 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:41.126050 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36084ac-0597-4d7c-a321-fc29efb662f9 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:41.127472 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 516/1031] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:11:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 17 19:11:41.132335 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72939766-b263-46d7-83eb-4b2520ecf8ce None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:41.132617 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72939766-b263-46d7-83eb-4b2520ecf8ce None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:41.132890 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72939766-b263-46d7-83eb-4b2520ecf8ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:41.149693 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-72939766-b263-46d7-83eb-4b2520ecf8ce None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:11:41.150860 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-72939766-b263-46d7-83eb-4b2520ecf8ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:11:41.206710 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0131e95-0315-4a91-a77f-731335f32d85 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:41.218096 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 517/1032] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:11:41 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 17 19:11:41.241273 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f602294-1b69-4cfe-8f92-afd393afdae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:41.241884 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72939766-b263-46d7-83eb-4b2520ecf8ce None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:41.242455 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f602294-1b69-4cfe-8f92-afd393afdae6 None admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:41.242919 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f602294-1b69-4cfe-8f92-afd393afdae6 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:41.243364 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f602294-1b69-4cfe-8f92-afd393afdae6 None admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:41.244488 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f602294-1b69-4cfe-8f92-afd393afdae6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:41.246862 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f602294-1b69-4cfe-8f92-afd393afdae6 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:41.335597 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72939766-b263-46d7-83eb-4b2520ecf8ce None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:41.341854 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 516/1033] 10.222.0.58 () {60 vars in 1132 bytes} [Tue May 17 19:11:41 2022] POST /identity/v3/users => generated 285 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:11:41.374033 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1659ec8d-c817-404c-bfcc-ac9d4f20e5c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:41.374869 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1659ec8d-c817-404c-bfcc-ac9d4f20e5c9 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:41.375244 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1659ec8d-c817-404c-bfcc-ac9d4f20e5c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:41.375757 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1659ec8d-c817-404c-bfcc-ac9d4f20e5c9 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:41.388209 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1659ec8d-c817-404c-bfcc-ac9d4f20e5c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 17 19:11:41.396536 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-1659ec8d-c817-404c-bfcc-ac9d4f20e5c9 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 17 19:11:41.396536 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:41.396536 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:11:41.396536 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:11:41.396536 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:11:41.396536 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:11:41.396536 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:11:41.396536 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:11:41.396536 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:11:41.396536 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:11:41.396536 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:11:41.396536 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:11:41.396536 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 17 19:11:41.396536 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) May 17 19:11:41.396536 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 17 19:11:41.396536 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 17 19:11:41.397585 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 17 19:11:41.397585 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 17 19:11:41.397585 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 17 19:11:41.397585 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 17 19:11:41.397585 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:11:41.397585 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:11:41.397585 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:11:41.397585 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:11:41.397585 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:11:41.397585 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:11:41.397585 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:11:41.397585 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:11:41.397585 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:11:41.397585 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:11:41.397585 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:11:41.397585 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:11:41.398688 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:11:41.398688 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:11:41.398688 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:11:41.398688 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:11:41.398688 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 17 19:11:41.398688 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 17 19:11:41.398688 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 17 19:11:41.398688 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 17 19:11:41.398688 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 17 19:11:41.398688 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 17 19:11:41.398688 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 17 19:11:41.398688 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:11:41.398688 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 518/1034] 10.222.0.58 () {56 vars in 1120 bytes} [Tue May 17 19:11:41 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 17 19:11:41.403894 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f602294-1b69-4cfe-8f92-afd393afdae6 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:41.508948 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f602294-1b69-4cfe-8f92-afd393afdae6 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:41.546343 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2769b266-0fc9-492d-bb92-023940b70a7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:41.547015 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2769b266-0fc9-492d-bb92-023940b70a7d None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:41.547293 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2769b266-0fc9-492d-bb92-023940b70a7d None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:41.547605 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2769b266-0fc9-492d-bb92-023940b70a7d None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:41.548384 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2769b266-0fc9-492d-bb92-023940b70a7d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:41.548681 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2769b266-0fc9-492d-bb92-023940b70a7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:41.550047 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2769b266-0fc9-492d-bb92-023940b70a7d None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:41.558800 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 517/1035] 10.222.0.58 () {56 vars in 1115 bytes} [Tue May 17 19:11:41 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:41.564648 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1659ec8d-c817-404c-bfcc-ac9d4f20e5c9 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:41.665519 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1659ec8d-c817-404c-bfcc-ac9d4f20e5c9 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:41.698682 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f553a0-e279-466b-b495-21c08b6b431c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:41.699727 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f553a0-e279-466b-b495-21c08b6b431c None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:41.701254 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f553a0-e279-466b-b495-21c08b6b431c None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:41.701254 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f553a0-e279-466b-b495-21c08b6b431c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:41.710588 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f553a0-e279-466b-b495-21c08b6b431c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:41.712084 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f553a0-e279-466b-b495-21c08b6b431c None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:41.728955 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-39f553a0-e279-466b-b495-21c08b6b431c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:11:41.728955 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:41.728955 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:11:41.728955 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:11:41.728955 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:11:41.728955 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:11:41.728955 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:11:41.728955 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:11:41.728955 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:11:41.728955 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:11:41.728955 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:11:41.728955 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:11:41.728955 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 17 19:11:41.728955 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 17 19:11:41.728955 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 17 19:11:41.729873 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 17 19:11:41.729873 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:11:41.729873 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:11:41.729873 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:11:41.729873 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:11:41.729873 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:11:41.729873 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:11:41.729873 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:11:41.729873 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:11:41.729873 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:11:41.729873 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:11:41.729873 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:11:41.729873 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:11:41.729873 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:11:41.729873 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:11:41.729873 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:11:41.729873 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:11:41.730759 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 17 19:11:41.730759 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 17 19:11:41.730759 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:11:41.730759 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:11:41.730759 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 519/1036] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:11:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:11:41.736735 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2769b266-0fc9-492d-bb92-023940b70a7d None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:41.830841 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2769b266-0fc9-492d-bb92-023940b70a7d None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:41.863309 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ab95fd-d129-4a8b-ac0b-a11b15c1a583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:41.863998 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ab95fd-d129-4a8b-ac0b-a11b15c1a583 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:41.864275 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ab95fd-d129-4a8b-ac0b-a11b15c1a583 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:41.864546 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ab95fd-d129-4a8b-ac0b-a11b15c1a583 None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:41.865264 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ab95fd-d129-4a8b-ac0b-a11b15c1a583 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:41.865564 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ab95fd-d129-4a8b-ac0b-a11b15c1a583 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:41.866923 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ab95fd-d129-4a8b-ac0b-a11b15c1a583 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:41.883684 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 518/1037] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 17 19:11:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:11:41.885164 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f553a0-e279-466b-b495-21c08b6b431c None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:41.986175 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f553a0-e279-466b-b495-21c08b6b431c None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:42.018122 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7585ca-b160-4aea-a4ef-2763014dafdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:42.018909 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7585ca-b160-4aea-a4ef-2763014dafdd None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:42.019184 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7585ca-b160-4aea-a4ef-2763014dafdd None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:42.019452 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7585ca-b160-4aea-a4ef-2763014dafdd None admin] PATH_INFO: `/v3/users/nova` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:42.026809 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7585ca-b160-4aea-a4ef-2763014dafdd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:42.028289 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7585ca-b160-4aea-a4ef-2763014dafdd None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:42.042347 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-9a7585ca-b160-4aea-a4ef-2763014dafdd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 17 19:11:42.042347 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:42.042347 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:11:42.042347 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:11:42.042347 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:11:42.042347 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:11:42.042347 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:11:42.042347 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:11:42.042347 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:11:42.042347 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:11:42.042347 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:11:42.042347 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:11:42.042347 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 17 19:11:42.042347 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_user(user_id) May 17 19:11:42.042347 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 17 19:11:42.042347 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 17 19:11:42.043308 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:11:42.043308 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:11:42.043308 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 17 19:11:42.043308 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 17 19:11:42.043308 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 17 19:11:42.043308 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 17 19:11:42.043308 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:11:42.043308 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:11:42.043308 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:11:42.043308 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:11:42.043308 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:11:42.043308 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:11:42.043308 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:11:42.043308 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:11:42.043308 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:11:42.043308 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:11:42.043308 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:11:42.043308 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:11:42.049907 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:11:42.049907 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:11:42.049907 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 17 19:11:42.049907 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 17 19:11:42.049907 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 17 19:11:42.049907 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 17 19:11:42.049907 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 17 19:11:42.049907 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 17 19:11:42.049907 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 17 19:11:42.049907 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:11:42.049907 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 520/1038] 10.222.0.58 () {56 vars in 1093 bytes} [Tue May 17 19:11:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 17 19:11:42.050519 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ab95fd-d129-4a8b-ac0b-a11b15c1a583 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:42.144394 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ab95fd-d129-4a8b-ac0b-a11b15c1a583 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:42.176680 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1914b27-f73c-44c0-b347-6982811d0c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:42.177342 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1914b27-f73c-44c0-b347-6982811d0c96 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:42.177620 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1914b27-f73c-44c0-b347-6982811d0c96 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:42.177886 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1914b27-f73c-44c0-b347-6982811d0c96 None admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:42.178708 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1914b27-f73c-44c0-b347-6982811d0c96 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:42.179011 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1914b27-f73c-44c0-b347-6982811d0c96 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:42.180435 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1914b27-f73c-44c0-b347-6982811d0c96 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:42.201959 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 519/1039] 10.222.0.58 () {56 vars in 1133 bytes} [Tue May 17 19:11:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:11:42.203602 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7585ca-b160-4aea-a4ef-2763014dafdd None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:42.299505 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7585ca-b160-4aea-a4ef-2763014dafdd None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:42.330848 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3468a1-75a1-4243-99bc-0a6ef09a8514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:42.331480 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3468a1-75a1-4243-99bc-0a6ef09a8514 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:42.331784 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3468a1-75a1-4243-99bc-0a6ef09a8514 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:42.332055 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3468a1-75a1-4243-99bc-0a6ef09a8514 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:42.395473 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3468a1-75a1-4243-99bc-0a6ef09a8514 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:42.396949 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3468a1-75a1-4243-99bc-0a6ef09a8514 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:42.494893 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 521/1040] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 17 19:11:42 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 17 19:11:42.496413 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1914b27-f73c-44c0-b347-6982811d0c96 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:42.594960 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1914b27-f73c-44c0-b347-6982811d0c96 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:42.627892 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130a5323-17a9-4004-a89d-5159ad2c92d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:42.628674 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130a5323-17a9-4004-a89d-5159ad2c92d5 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:42.628950 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130a5323-17a9-4004-a89d-5159ad2c92d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:42.629220 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130a5323-17a9-4004-a89d-5159ad2c92d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:42.638778 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130a5323-17a9-4004-a89d-5159ad2c92d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:42.640402 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130a5323-17a9-4004-a89d-5159ad2c92d5 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:42.658353 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-130a5323-17a9-4004-a89d-5159ad2c92d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:11:42.658353 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:42.658353 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:11:42.658353 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:11:42.658353 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:11:42.658353 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:11:42.658353 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:11:42.658353 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:11:42.658353 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:11:42.658353 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:11:42.658353 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:11:42.658353 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:11:42.658353 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 17 19:11:42.658353 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 17 19:11:42.658353 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 17 19:11:42.659295 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 17 19:11:42.659295 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:11:42.659295 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:11:42.659295 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:11:42.659295 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:11:42.659295 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:11:42.659295 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:11:42.659295 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:11:42.659295 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:11:42.659295 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:11:42.659295 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:11:42.659295 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:11:42.659295 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:11:42.659295 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:11:42.659295 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:11:42.659295 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:11:42.659295 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:11:42.660801 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 17 19:11:42.660801 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 17 19:11:42.660801 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:11:42.660801 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:11:42.662994 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 520/1041] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:11:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 17 19:11:42.665319 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3468a1-75a1-4243-99bc-0a6ef09a8514 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:42.761497 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3468a1-75a1-4243-99bc-0a6ef09a8514 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:42.793584 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c998bf-5ccd-468a-857d-b15603143792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:42.794239 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c998bf-5ccd-468a-857d-b15603143792 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:42.794517 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c998bf-5ccd-468a-857d-b15603143792 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:42.794787 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c998bf-5ccd-468a-857d-b15603143792 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:42.795515 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c998bf-5ccd-468a-857d-b15603143792 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:42.795819 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c998bf-5ccd-468a-857d-b15603143792 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:42.797179 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c998bf-5ccd-468a-857d-b15603143792 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:42.811535 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 522/1042] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 17 19:11:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:42.814579 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130a5323-17a9-4004-a89d-5159ad2c92d5 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:42.916466 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130a5323-17a9-4004-a89d-5159ad2c92d5 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:42.950598 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4386f7bc-c53c-4de9-a36c-12210df46ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:42.951397 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4386f7bc-c53c-4de9-a36c-12210df46ff9 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:42.951704 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4386f7bc-c53c-4de9-a36c-12210df46ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:42.951999 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4386f7bc-c53c-4de9-a36c-12210df46ff9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:42.959223 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4386f7bc-c53c-4de9-a36c-12210df46ff9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:42.960689 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4386f7bc-c53c-4de9-a36c-12210df46ff9 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:42.974246 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-4386f7bc-c53c-4de9-a36c-12210df46ff9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 17 19:11:42.974246 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:42.974246 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:11:42.974246 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:11:42.974246 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:11:42.974246 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:11:42.974246 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:11:42.974246 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:11:42.974246 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:11:42.974246 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:11:42.974246 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:11:42.974246 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:11:42.974246 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 17 19:11:42.974246 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(project_id) May 17 19:11:42.974246 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 17 19:11:42.975174 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 17 19:11:42.975174 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:11:42.975174 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:11:42.975174 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:11:42.975174 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:11:42.975174 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:11:42.975174 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:11:42.975174 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:11:42.975174 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:11:42.975174 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:11:42.975174 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:11:42.975174 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:11:42.975174 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:11:42.975174 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:11:42.975174 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:11:42.975174 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:11:42.975174 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:11:42.980844 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 17 19:11:42.980844 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 17 19:11:42.980844 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 17 19:11:42.980844 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 17 19:11:42.980844 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 17 19:11:42.980844 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 17 19:11:42.980844 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 17 19:11:42.980844 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:11:42.980844 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 521/1043] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 17 19:11:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:11:42.983216 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c998bf-5ccd-468a-857d-b15603143792 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:43.040249 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae4d81f-c981-4b65-b1e4-0c27ac113b08 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:43.040427 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae4d81f-c981-4b65-b1e4-0c27ac113b08 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:43.040799 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae4d81f-c981-4b65-b1e4-0c27ac113b08 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:43.042403 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 522/1044] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:11:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:11:43.047193 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1b6bd1-c889-477d-a20b-9c3c07d14017 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:43.047481 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1b6bd1-c889-477d-a20b-9c3c07d14017 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:43.047775 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1b6bd1-c889-477d-a20b-9c3c07d14017 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:43.065118 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-6c1b6bd1-c889-477d-a20b-9c3c07d14017 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:11:43.066297 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-6c1b6bd1-c889-477d-a20b-9c3c07d14017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:11:43.084893 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c998bf-5ccd-468a-857d-b15603143792 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:43.119358 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59723efd-042a-4b67-8619-1fddeea334ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:43.120107 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59723efd-042a-4b67-8619-1fddeea334ab None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:43.120326 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59723efd-042a-4b67-8619-1fddeea334ab None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:43.120595 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59723efd-042a-4b67-8619-1fddeea334ab None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:43.121376 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59723efd-042a-4b67-8619-1fddeea334ab None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:43.121681 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59723efd-042a-4b67-8619-1fddeea334ab None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:43.123190 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59723efd-042a-4b67-8619-1fddeea334ab None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:43.133508 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 523/1045] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:11:43 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 17 19:11:43.140583 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 523/1046] 10.222.0.58 () {56 vars in 1148 bytes} [Tue May 17 19:11:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:43.141548 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1b6bd1-c889-477d-a20b-9c3c07d14017 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:43.157577 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59723efd-042a-4b67-8619-1fddeea334ab None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:43.239662 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1b6bd1-c889-477d-a20b-9c3c07d14017 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:43.256894 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59723efd-042a-4b67-8619-1fddeea334ab None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:43.272974 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b732b6b-d0b4-4646-a0ec-a15ffa8d8cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:43.273852 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b732b6b-d0b4-4646-a0ec-a15ffa8d8cf4 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:43.274284 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b732b6b-d0b4-4646-a0ec-a15ffa8d8cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:43.274707 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b732b6b-d0b4-4646-a0ec-a15ffa8d8cf4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:43.275707 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b732b6b-d0b4-4646-a0ec-a15ffa8d8cf4 None admin] RBAC: Adding query filter params (role.id=897862af1c83498f86d0dfdf48bea276, scope.project.id=1a9d3a0631284c9fb113266036993e2d, user.id=65c1f878b0dc4d63a4f5dcb12fac1dc1) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:43.276163 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b732b6b-d0b4-4646-a0ec-a15ffa8d8cf4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:43.277755 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b732b6b-d0b4-4646-a0ec-a15ffa8d8cf4 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:43.283314 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 524/1047] 10.222.0.58 () {56 vars in 1374 bytes} [Tue May 17 19:11:43 2022] GET /identity/v3/role_assignments?user.id=65c1f878b0dc4d63a4f5dcb12fac1dc1&scope.project.id=1a9d3a0631284c9fb113266036993e2d&role.id=897862af1c83498f86d0dfdf48bea276 => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:43.290376 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d0cb35-0691-4941-b5b2-6fedc6726422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:43.291561 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d0cb35-0691-4941-b5b2-6fedc6726422 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:43.292066 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d0cb35-0691-4941-b5b2-6fedc6726422 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:43.292575 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d0cb35-0691-4941-b5b2-6fedc6726422 None admin] PATH_INFO: `/v3/services/image` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:43.293442 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d0cb35-0691-4941-b5b2-6fedc6726422 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:43.295090 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d0cb35-0691-4941-b5b2-6fedc6726422 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:43.308497 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-52d0cb35-0691-4941-b5b2-6fedc6726422 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 17 19:11:43.308497 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:43.308497 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 17 19:11:43.308497 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 17 19:11:43.308497 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 17 19:11:43.308497 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 17 19:11:43.308497 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 17 19:11:43.308497 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 17 19:11:43.308497 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 17 19:11:43.308497 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:11:43.308497 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 17 19:11:43.308497 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:11:43.308497 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:43.308497 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:11:43.308497 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:11:43.308497 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:11:43.308497 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:11:43.309582 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:11:43.309582 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:11:43.309582 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:11:43.309582 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:11:43.309582 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:11:43.309582 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:11:43.309582 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 17 19:11:43.309582 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(service_id) May 17 19:11:43.309582 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 17 19:11:43.309582 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 17 19:11:43.309582 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:11:43.309582 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:11:43.309582 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:11:43.309582 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:11:43.309582 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:11:43.309582 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:11:43.310441 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:11:43.310441 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:11:43.310441 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:11:43.310441 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:11:43.310441 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:11:43.310441 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:11:43.310441 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:11:43.310441 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:11:43.310441 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:11:43.310441 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:11:43.310441 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 17 19:11:43.310441 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 17 19:11:43.310441 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 17 19:11:43.310441 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:11:43.318338 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 524/1048] 10.222.0.58 () {56 vars in 1105 bytes} [Tue May 17 19:11:43 2022] GET /identity/v3/services/image => generated 86 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 17 19:11:43.319507 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b732b6b-d0b4-4646-a0ec-a15ffa8d8cf4 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:43.424485 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b732b6b-d0b4-4646-a0ec-a15ffa8d8cf4 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:43.460958 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d5c190-468b-4b12-bac5-0b1d36322450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:43.461623 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d5c190-468b-4b12-bac5-0b1d36322450 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:43.461904 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d5c190-468b-4b12-bac5-0b1d36322450 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:43.462173 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d5c190-468b-4b12-bac5-0b1d36322450 None admin] PATH_INFO: `/v3/services` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:43.462878 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d5c190-468b-4b12-bac5-0b1d36322450 None admin] RBAC: Adding query filter params (name=image) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:43.463183 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d5c190-468b-4b12-bac5-0b1d36322450 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:43.464576 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d5c190-468b-4b12-bac5-0b1d36322450 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:43.470398 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 525/1049] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:11:43 2022] GET /identity/v3/services?name=image => generated 122 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:43.476560 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d0cb35-0691-4941-b5b2-6fedc6726422 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:43.573584 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d0cb35-0691-4941-b5b2-6fedc6726422 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:43.604443 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7213799-0dc8-4399-9705-d0c5137d7fe9 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:43.605006 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7213799-0dc8-4399-9705-d0c5137d7fe9 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:43.605500 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7213799-0dc8-4399-9705-d0c5137d7fe9 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:43.607310 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d3b05c-5e10-4230-9ce2-5fe76881183f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:43.607774 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 526/1050] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:11:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 17 19:11:43.607966 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d3b05c-5e10-4230-9ce2-5fe76881183f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:43.608241 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d3b05c-5e10-4230-9ce2-5fe76881183f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:43.608510 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d3b05c-5e10-4230-9ce2-5fe76881183f None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:43.609198 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d3b05c-5e10-4230-9ce2-5fe76881183f None admin] RBAC: Adding query filter params (type=image) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:43.609497 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d3b05c-5e10-4230-9ce2-5fe76881183f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:43.611402 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d3b05c-5e10-4230-9ce2-5fe76881183f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:43.612834 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340c649e-c422-491b-a553-3a2fcb449673 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:43.613413 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340c649e-c422-491b-a553-3a2fcb449673 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:43.613782 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340c649e-c422-491b-a553-3a2fcb449673 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:43.616420 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 525/1051] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:11:43 2022] GET /identity/v3/services?type=image => generated 350 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:11:43.622283 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d3b05c-5e10-4230-9ce2-5fe76881183f None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:43.630907 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-340c649e-c422-491b-a553-3a2fcb449673 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:11:43.632237 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-340c649e-c422-491b-a553-3a2fcb449673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:11:43.699122 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 527/1052] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:11:43 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 17 19:11:43.719720 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d3b05c-5e10-4230-9ce2-5fe76881183f None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:43.722814 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340c649e-c422-491b-a553-3a2fcb449673 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:43.753276 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a819d0db-10c3-490e-9579-710f8820dd64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:43.754621 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a819d0db-10c3-490e-9579-710f8820dd64 None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:43.754990 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a819d0db-10c3-490e-9579-710f8820dd64 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:43.755357 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a819d0db-10c3-490e-9579-710f8820dd64 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:43.756334 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a819d0db-10c3-490e-9579-710f8820dd64 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:43.757716 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a819d0db-10c3-490e-9579-710f8820dd64 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:43.819272 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340c649e-c422-491b-a553-3a2fcb449673 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:43.852641 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9dfe9f-3a4d-4488-8c4f-b532bc8fa4e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:43.853353 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9dfe9f-3a4d-4488-8c4f-b532bc8fa4e5 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:43.853745 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9dfe9f-3a4d-4488-8c4f-b532bc8fa4e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:43.854022 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9dfe9f-3a4d-4488-8c4f-b532bc8fa4e5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:43.864189 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9dfe9f-3a4d-4488-8c4f-b532bc8fa4e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 17 19:11:43.871699 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-9f9dfe9f-3a4d-4488-8c4f-b532bc8fa4e5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 17 19:11:43.871699 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:43.871699 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:11:43.871699 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:11:43.871699 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:11:43.871699 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:11:43.871699 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:11:43.871699 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:11:43.871699 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:11:43.871699 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:11:43.871699 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:11:43.871699 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:11:43.871699 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 17 19:11:43.871699 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) May 17 19:11:43.871699 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 17 19:11:43.871699 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 17 19:11:43.872683 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 17 19:11:43.872683 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 17 19:11:43.872683 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 17 19:11:43.872683 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 17 19:11:43.872683 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:11:43.872683 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:11:43.872683 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:11:43.872683 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:11:43.872683 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:11:43.872683 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:11:43.872683 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:11:43.872683 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:11:43.872683 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:11:43.872683 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:11:43.872683 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:11:43.872683 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:11:43.873527 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:11:43.873527 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:11:43.873527 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:11:43.873527 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:11:43.873527 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 17 19:11:43.873527 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 17 19:11:43.873527 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 17 19:11:43.873527 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 17 19:11:43.873527 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 17 19:11:43.873527 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 17 19:11:43.873527 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 17 19:11:43.873527 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:11:43.873527 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 528/1053] 10.222.0.58 () {56 vars in 1096 bytes} [Tue May 17 19:11:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 17 19:11:43.878648 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9dfe9f-3a4d-4488-8c4f-b532bc8fa4e5 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:43.881561 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 526/1054] 10.222.0.58 () {60 vars in 1144 bytes} [Tue May 17 19:11:43 2022] POST /identity/v3/endpoints => generated 327 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 17 19:11:43.985981 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9dfe9f-3a4d-4488-8c4f-b532bc8fa4e5 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:44.026690 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be78c27-f8ee-495e-a2b5-78b008c1dff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:44.027521 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be78c27-f8ee-495e-a2b5-78b008c1dff9 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:44.027894 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be78c27-f8ee-495e-a2b5-78b008c1dff9 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:44.028259 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be78c27-f8ee-495e-a2b5-78b008c1dff9 None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:44.029103 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be78c27-f8ee-495e-a2b5-78b008c1dff9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:44.029651 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be78c27-f8ee-495e-a2b5-78b008c1dff9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:44.031058 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be78c27-f8ee-495e-a2b5-78b008c1dff9 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:44.041410 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 529/1055] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 17 19:11:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:44.046980 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a819d0db-10c3-490e-9579-710f8820dd64 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:44.146911 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a819d0db-10c3-490e-9579-710f8820dd64 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:44.178017 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc11084-d019-4552-83b5-ccbe132b9f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:44.178875 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc11084-d019-4552-83b5-ccbe132b9f72 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:44.179147 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc11084-d019-4552-83b5-ccbe132b9f72 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:44.179415 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc11084-d019-4552-83b5-ccbe132b9f72 None admin] PATH_INFO: `/v3/users/bc78d499848447e28a145b13c385fa63` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:44.192596 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc11084-d019-4552-83b5-ccbe132b9f72 None admin] RBAC: Authorizing `identity:get_user(user_id=bc78d499848447e28a145b13c385fa63)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:44.194041 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc11084-d019-4552-83b5-ccbe132b9f72 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:44.195872 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 527/1056] 10.222.0.58 () {56 vars in 1177 bytes} [Tue May 17 19:11:44 2022] GET /identity/v3/users/bc78d499848447e28a145b13c385fa63 => generated 285 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:44.201244 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be78c27-f8ee-495e-a2b5-78b008c1dff9 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:44.299367 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be78c27-f8ee-495e-a2b5-78b008c1dff9 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:44.332510 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66277fae-012e-46d1-8107-c911e351651e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:44.333157 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66277fae-012e-46d1-8107-c911e351651e None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:44.333444 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66277fae-012e-46d1-8107-c911e351651e None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:44.333718 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66277fae-012e-46d1-8107-c911e351651e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:44.398697 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66277fae-012e-46d1-8107-c911e351651e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:44.400136 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66277fae-012e-46d1-8107-c911e351651e None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:44.500720 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 530/1057] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 17 19:11:44 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 17 19:11:44.502552 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc11084-d019-4552-83b5-ccbe132b9f72 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:44.597434 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc11084-d019-4552-83b5-ccbe132b9f72 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:44.630051 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdef6c50-12fa-49b3-8e77-0bcd450ca813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:44.630897 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdef6c50-12fa-49b3-8e77-0bcd450ca813 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:44.631169 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdef6c50-12fa-49b3-8e77-0bcd450ca813 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:44.631433 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdef6c50-12fa-49b3-8e77-0bcd450ca813 None admin] PATH_INFO: `/v3/projects/2ff0f1d671ce48bab664650af5095826` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:44.632612 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdef6c50-12fa-49b3-8e77-0bcd450ca813 None admin] RBAC: Authorizing `identity:get_project(project_id=2ff0f1d671ce48bab664650af5095826)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:44.634034 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdef6c50-12fa-49b3-8e77-0bcd450ca813 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:44.638690 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 528/1058] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 17 19:11:44 2022] GET /identity/v3/projects/2ff0f1d671ce48bab664650af5095826 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:44.641657 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66277fae-012e-46d1-8107-c911e351651e None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:44.734987 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66277fae-012e-46d1-8107-c911e351651e None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:44.768517 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe42972-ac1d-4112-b732-0cbace1d0eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:44.769168 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe42972-ac1d-4112-b732-0cbace1d0eb9 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:44.769445 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe42972-ac1d-4112-b732-0cbace1d0eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:44.769715 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe42972-ac1d-4112-b732-0cbace1d0eb9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:44.770533 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe42972-ac1d-4112-b732-0cbace1d0eb9 None admin] RBAC: Adding query filter params (role.id=60af5088793e4379b88821a86552a344, scope.project.id=2ff0f1d671ce48bab664650af5095826, user.id=bc78d499848447e28a145b13c385fa63) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:44.770837 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe42972-ac1d-4112-b732-0cbace1d0eb9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:44.772283 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe42972-ac1d-4112-b732-0cbace1d0eb9 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:44.783584 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 531/1059] 10.222.0.58 () {56 vars in 1374 bytes} [Tue May 17 19:11:44 2022] GET /identity/v3/role_assignments?user.id=bc78d499848447e28a145b13c385fa63&scope.project.id=2ff0f1d671ce48bab664650af5095826&role.id=60af5088793e4379b88821a86552a344 => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:11:45.490190 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3ea7f0-1f53-43fc-8aea-2c9015646184 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:45.490716 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3ea7f0-1f53-43fc-8aea-2c9015646184 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:45.490716 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3ea7f0-1f53-43fc-8aea-2c9015646184 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:45.492251 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 529/1060] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:11:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:11:45.497419 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2576b250-7d9f-44f9-bd90-424bf4c01c96 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:45.497819 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2576b250-7d9f-44f9-bd90-424bf4c01c96 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:45.498005 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2576b250-7d9f-44f9-bd90-424bf4c01c96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:45.514608 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-2576b250-7d9f-44f9-bd90-424bf4c01c96 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:11:45.515805 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-2576b250-7d9f-44f9-bd90-424bf4c01c96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:11:45.580703 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 532/1061] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:11:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 17 19:11:45.603334 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3ea7f0-1f53-43fc-8aea-2c9015646184 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:45.700632 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3ea7f0-1f53-43fc-8aea-2c9015646184 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:45.734023 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfad950-6c47-4229-b485-34ea65100a97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:45.734818 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfad950-6c47-4229-b485-34ea65100a97 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:45.735096 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfad950-6c47-4229-b485-34ea65100a97 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:45.735424 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfad950-6c47-4229-b485-34ea65100a97 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:45.745687 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfad950-6c47-4229-b485-34ea65100a97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 17 19:11:45.753655 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-1dfad950-6c47-4229-b485-34ea65100a97 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 17 19:11:45.753655 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:45.753655 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:11:45.753655 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:11:45.753655 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:11:45.753655 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:11:45.753655 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:11:45.753655 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:11:45.753655 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:11:45.753655 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:11:45.753655 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:11:45.753655 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:11:45.753655 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 17 19:11:45.753655 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) May 17 19:11:45.753655 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 17 19:11:45.753655 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 17 19:11:45.754629 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 17 19:11:45.754629 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 17 19:11:45.754629 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 17 19:11:45.754629 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 17 19:11:45.754629 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:11:45.754629 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:11:45.754629 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:11:45.754629 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:11:45.754629 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:11:45.754629 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:11:45.754629 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:11:45.754629 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:11:45.754629 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:11:45.754629 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:11:45.754629 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:11:45.754629 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:11:45.755512 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:11:45.755512 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:11:45.755512 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:11:45.755512 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:11:45.755512 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 17 19:11:45.755512 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 17 19:11:45.755512 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 17 19:11:45.755512 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 17 19:11:45.755512 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 17 19:11:45.755512 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 17 19:11:45.755512 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 17 19:11:45.755512 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:11:45.759572 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 530/1062] 10.222.0.58 () {56 vars in 1120 bytes} [Tue May 17 19:11:45 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:11:45.761042 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2576b250-7d9f-44f9-bd90-424bf4c01c96 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:45.854467 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2576b250-7d9f-44f9-bd90-424bf4c01c96 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:45.889086 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625ccae1-3c67-42f9-9eb0-fc5dfebc9a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:45.889754 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625ccae1-3c67-42f9-9eb0-fc5dfebc9a5f None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:45.890030 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625ccae1-3c67-42f9-9eb0-fc5dfebc9a5f None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:45.890297 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625ccae1-3c67-42f9-9eb0-fc5dfebc9a5f None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:45.891034 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625ccae1-3c67-42f9-9eb0-fc5dfebc9a5f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:45.891333 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625ccae1-3c67-42f9-9eb0-fc5dfebc9a5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:45.892731 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625ccae1-3c67-42f9-9eb0-fc5dfebc9a5f None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:45.900605 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 533/1063] 10.222.0.58 () {56 vars in 1115 bytes} [Tue May 17 19:11:45 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:45.906238 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfad950-6c47-4229-b485-34ea65100a97 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:46.013980 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfad950-6c47-4229-b485-34ea65100a97 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:46.046887 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f6b325-5a84-49a4-a16a-602710bc30ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:46.047697 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f6b325-5a84-49a4-a16a-602710bc30ac None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:46.048006 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f6b325-5a84-49a4-a16a-602710bc30ac None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:46.048275 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f6b325-5a84-49a4-a16a-602710bc30ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:46.058429 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f6b325-5a84-49a4-a16a-602710bc30ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:46.059907 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f6b325-5a84-49a4-a16a-602710bc30ac None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:46.076433 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-99f6b325-5a84-49a4-a16a-602710bc30ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:11:46.076433 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:46.076433 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:11:46.076433 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:11:46.076433 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:11:46.076433 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:11:46.076433 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:11:46.076433 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:11:46.076433 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:11:46.076433 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:11:46.076433 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:11:46.076433 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:11:46.076433 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 17 19:11:46.076433 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 17 19:11:46.076433 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 17 19:11:46.077343 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 17 19:11:46.077343 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:11:46.077343 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:11:46.077343 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:11:46.077343 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:11:46.077343 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:11:46.077343 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:11:46.077343 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:11:46.077343 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:11:46.077343 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:11:46.077343 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:11:46.077343 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:11:46.077343 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:11:46.077343 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:11:46.077343 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:11:46.077343 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:11:46.077343 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:11:46.078330 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 17 19:11:46.078330 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 17 19:11:46.078330 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:11:46.078330 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:11:46.078330 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 531/1064] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:11:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:11:46.083884 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625ccae1-3c67-42f9-9eb0-fc5dfebc9a5f None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:46.122379 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31444bc0-f684-40bb-a572-62a64870f8ca None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:46.122666 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31444bc0-f684-40bb-a572-62a64870f8ca None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:46.122933 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31444bc0-f684-40bb-a572-62a64870f8ca None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:46.124568 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 532/1065] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:11:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 17 19:11:46.129286 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8555621e-7ddd-4900-96aa-20b7fd900742 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:46.129561 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8555621e-7ddd-4900-96aa-20b7fd900742 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:46.129831 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8555621e-7ddd-4900-96aa-20b7fd900742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:46.146516 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-8555621e-7ddd-4900-96aa-20b7fd900742 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:11:46.147723 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-8555621e-7ddd-4900-96aa-20b7fd900742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:11:46.180462 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625ccae1-3c67-42f9-9eb0-fc5dfebc9a5f None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:46.212716 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e21b85-285b-43d7-a9c5-30aed913e905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:46.213359 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e21b85-285b-43d7-a9c5-30aed913e905 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:46.213637 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e21b85-285b-43d7-a9c5-30aed913e905 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:46.213906 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e21b85-285b-43d7-a9c5-30aed913e905 None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:46.214606 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e21b85-285b-43d7-a9c5-30aed913e905 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:46.214904 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e21b85-285b-43d7-a9c5-30aed913e905 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:46.223554 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 533/1066] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:11:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 17 19:11:46.223692 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e21b85-285b-43d7-a9c5-30aed913e905 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:46.229092 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 534/1067] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 17 19:11:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:46.234626 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8555621e-7ddd-4900-96aa-20b7fd900742 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:46.238537 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e21b85-285b-43d7-a9c5-30aed913e905 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:46.332368 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e21b85-285b-43d7-a9c5-30aed913e905 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:46.333738 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8555621e-7ddd-4900-96aa-20b7fd900742 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:46.365377 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ab24a5-2f65-4588-ab71-d5d4d753d14d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:46.366015 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ab24a5-2f65-4588-ab71-d5d4d753d14d None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:46.366323 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ab24a5-2f65-4588-ab71-d5d4d753d14d None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:46.366557 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ab24a5-2f65-4588-ab71-d5d4d753d14d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:46.366924 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee86cb2-8da1-4d19-a27e-26ada55190de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:46.367737 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee86cb2-8da1-4d19-a27e-26ada55190de None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:46.368017 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee86cb2-8da1-4d19-a27e-26ada55190de None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:46.368289 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee86cb2-8da1-4d19-a27e-26ada55190de None admin] PATH_INFO: `/v3/users/nova` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:46.375707 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee86cb2-8da1-4d19-a27e-26ada55190de None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:46.377189 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee86cb2-8da1-4d19-a27e-26ada55190de None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:46.391358 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-eee86cb2-8da1-4d19-a27e-26ada55190de None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 17 19:11:46.391358 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:46.391358 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:11:46.391358 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:11:46.391358 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:11:46.391358 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:11:46.391358 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:11:46.391358 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:11:46.391358 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:11:46.391358 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:11:46.391358 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:11:46.391358 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:11:46.391358 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 17 19:11:46.391358 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_user(user_id) May 17 19:11:46.391358 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 17 19:11:46.391358 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 17 19:11:46.392594 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:11:46.392594 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:11:46.392594 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 17 19:11:46.392594 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 17 19:11:46.392594 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 17 19:11:46.392594 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 17 19:11:46.392594 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:11:46.392594 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:11:46.392594 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:11:46.392594 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:11:46.392594 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:11:46.392594 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:11:46.392594 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:11:46.392594 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:11:46.392594 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:11:46.392594 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:11:46.392594 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:11:46.392594 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:11:46.393580 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:11:46.393580 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:11:46.393580 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 17 19:11:46.393580 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 17 19:11:46.393580 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 17 19:11:46.393580 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 17 19:11:46.393580 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 17 19:11:46.393580 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 17 19:11:46.393580 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 17 19:11:46.393580 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:11:46.393580 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 534/1068] 10.222.0.58 () {56 vars in 1093 bytes} [Tue May 17 19:11:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 17 19:11:46.398515 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee86cb2-8da1-4d19-a27e-26ada55190de None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:46.435458 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ab24a5-2f65-4588-ab71-d5d4d753d14d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:46.436940 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ab24a5-2f65-4588-ab71-d5d4d753d14d None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:46.495116 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee86cb2-8da1-4d19-a27e-26ada55190de None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:46.529276 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48252ef7-d7f7-4737-9f69-995b8bce3cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:46.530187 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48252ef7-d7f7-4737-9f69-995b8bce3cff None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:46.530459 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48252ef7-d7f7-4737-9f69-995b8bce3cff None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:46.530729 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48252ef7-d7f7-4737-9f69-995b8bce3cff None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:46.531579 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48252ef7-d7f7-4737-9f69-995b8bce3cff None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:46.531883 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48252ef7-d7f7-4737-9f69-995b8bce3cff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:46.533851 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48252ef7-d7f7-4737-9f69-995b8bce3cff None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:46.535640 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 535/1069] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 17 19:11:46 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 17 19:11:46.541008 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ab24a5-2f65-4588-ab71-d5d4d753d14d None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:46.557367 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 535/1070] 10.222.0.58 () {56 vars in 1133 bytes} [Tue May 17 19:11:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:11:46.558408 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48252ef7-d7f7-4737-9f69-995b8bce3cff None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:46.644426 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ab24a5-2f65-4588-ab71-d5d4d753d14d None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:46.663947 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48252ef7-d7f7-4737-9f69-995b8bce3cff None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:46.681131 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9913bc-d0ea-403a-83b5-e3701088d53b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:46.681927 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9913bc-d0ea-403a-83b5-e3701088d53b None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:46.682220 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9913bc-d0ea-403a-83b5-e3701088d53b None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:46.682513 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9913bc-d0ea-403a-83b5-e3701088d53b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:46.692532 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9913bc-d0ea-403a-83b5-e3701088d53b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:46.693970 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9913bc-d0ea-403a-83b5-e3701088d53b None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:46.698075 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c47429e-aa46-48ce-969f-40d5771d931f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:46.698871 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c47429e-aa46-48ce-969f-40d5771d931f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:46.699147 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c47429e-aa46-48ce-969f-40d5771d931f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:46.699415 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c47429e-aa46-48ce-969f-40d5771d931f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:46.710206 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c47429e-aa46-48ce-969f-40d5771d931f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:46.711656 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-ce9913bc-d0ea-403a-83b5-e3701088d53b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:11:46.711656 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:46.711656 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:11:46.711656 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:11:46.711656 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:11:46.711656 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:11:46.711656 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:11:46.711656 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:11:46.711656 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:11:46.711656 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:11:46.711656 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:11:46.711656 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:11:46.711656 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 17 19:11:46.711656 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 17 19:11:46.711656 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 17 19:11:46.712594 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 17 19:11:46.712594 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:11:46.712594 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:11:46.712594 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:11:46.712594 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:11:46.712594 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:11:46.712594 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:11:46.712594 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:11:46.712594 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:11:46.712594 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:11:46.712594 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:11:46.712594 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:11:46.712594 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:11:46.712594 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:11:46.712594 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:11:46.712594 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:11:46.712594 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:11:46.713586 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 17 19:11:46.713586 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 17 19:11:46.713586 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:11:46.713586 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:11:46.713826 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c47429e-aa46-48ce-969f-40d5771d931f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:46.713902 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 536/1071] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:11:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 17 19:11:46.718507 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9913bc-d0ea-403a-83b5-e3701088d53b None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:46.732045 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-2c47429e-aa46-48ce-969f-40d5771d931f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:11:46.732045 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:46.732045 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:11:46.732045 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:11:46.732045 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:11:46.732045 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:11:46.732045 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:11:46.732045 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:11:46.732045 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:11:46.732045 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:11:46.732045 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:11:46.732045 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:11:46.732045 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 17 19:11:46.732045 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 17 19:11:46.732045 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 17 19:11:46.732943 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 17 19:11:46.732943 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:11:46.732943 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:11:46.732943 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:11:46.732943 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:11:46.732943 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:11:46.732943 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:11:46.732943 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:11:46.732943 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:11:46.732943 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:11:46.732943 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:11:46.732943 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:11:46.732943 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:11:46.732943 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:11:46.732943 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:11:46.732943 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:11:46.732943 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:11:46.733909 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 17 19:11:46.733909 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 17 19:11:46.733909 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:11:46.733909 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:11:46.733909 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 536/1072] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:11:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:11:46.738675 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c47429e-aa46-48ce-969f-40d5771d931f None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:46.820867 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9913bc-d0ea-403a-83b5-e3701088d53b None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:46.839140 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c47429e-aa46-48ce-969f-40d5771d931f None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:46.862678 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e28a7e-e2ee-47f7-92ab-955cd81fc2b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:46.863348 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e28a7e-e2ee-47f7-92ab-955cd81fc2b6 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:46.863676 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e28a7e-e2ee-47f7-92ab-955cd81fc2b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:46.863943 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e28a7e-e2ee-47f7-92ab-955cd81fc2b6 None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:46.864670 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e28a7e-e2ee-47f7-92ab-955cd81fc2b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:46.864975 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e28a7e-e2ee-47f7-92ab-955cd81fc2b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:46.866337 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e28a7e-e2ee-47f7-92ab-955cd81fc2b6 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:46.877286 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b299805a-2a53-4bd2-9b1d-7d7168a785f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:46.877946 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b299805a-2a53-4bd2-9b1d-7d7168a785f3 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:46.878226 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b299805a-2a53-4bd2-9b1d-7d7168a785f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:46.878498 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b299805a-2a53-4bd2-9b1d-7d7168a785f3 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:46.879220 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b299805a-2a53-4bd2-9b1d-7d7168a785f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:46.880027 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b299805a-2a53-4bd2-9b1d-7d7168a785f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:46.881429 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b299805a-2a53-4bd2-9b1d-7d7168a785f3 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:46.887948 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 537/1073] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 17 19:11:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:11:46.894020 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 537/1074] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 17 19:11:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:11:46.899952 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e28a7e-e2ee-47f7-92ab-955cd81fc2b6 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:46.989942 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18303f4b-f202-4987-8114-e0c520a17260 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:46.990285 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18303f4b-f202-4987-8114-e0c520a17260 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:46.990497 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18303f4b-f202-4987-8114-e0c520a17260 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:46.992063 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 538/1075] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:11:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:11:46.997513 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b18a25-ac3e-4704-83b7-a36aec855aae None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:46.997802 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b18a25-ac3e-4704-83b7-a36aec855aae None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:46.998084 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b18a25-ac3e-4704-83b7-a36aec855aae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:47.015220 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-d8b18a25-ac3e-4704-83b7-a36aec855aae None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:11:47.017010 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d8b18a25-ac3e-4704-83b7-a36aec855aae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:11:47.019133 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e28a7e-e2ee-47f7-92ab-955cd81fc2b6 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:47.057114 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c953237-fb9a-410a-ad62-e73ddd94b644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:47.057911 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c953237-fb9a-410a-ad62-e73ddd94b644 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:47.058189 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c953237-fb9a-410a-ad62-e73ddd94b644 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:47.058457 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c953237-fb9a-410a-ad62-e73ddd94b644 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:47.065852 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c953237-fb9a-410a-ad62-e73ddd94b644 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:47.067288 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c953237-fb9a-410a-ad62-e73ddd94b644 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:47.083243 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-3c953237-fb9a-410a-ad62-e73ddd94b644 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 17 19:11:47.083243 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:47.083243 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:11:47.083243 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:11:47.083243 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:11:47.083243 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:11:47.083243 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:11:47.083243 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:11:47.083243 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:11:47.083243 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:11:47.083243 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:11:47.083243 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:11:47.083243 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 17 19:11:47.083243 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(project_id) May 17 19:11:47.083243 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 17 19:11:47.084226 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 17 19:11:47.084226 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:11:47.084226 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:11:47.084226 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:11:47.084226 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:11:47.084226 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:11:47.084226 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:11:47.084226 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:11:47.084226 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:11:47.084226 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:11:47.084226 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:11:47.084226 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:11:47.084226 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:11:47.084226 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:11:47.084226 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:11:47.084226 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:11:47.084226 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:11:47.085315 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 17 19:11:47.085315 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 17 19:11:47.085315 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 17 19:11:47.085315 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 17 19:11:47.085315 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 17 19:11:47.085315 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 17 19:11:47.085315 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 17 19:11:47.085315 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:11:47.085315 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 538/1076] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 17 19:11:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 17 19:11:47.091756 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c953237-fb9a-410a-ad62-e73ddd94b644 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:47.092905 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 539/1077] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:11:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 17 19:11:47.115022 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b18a25-ac3e-4704-83b7-a36aec855aae None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:47.197421 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c953237-fb9a-410a-ad62-e73ddd94b644 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:47.223301 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b18a25-ac3e-4704-83b7-a36aec855aae None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:47.230198 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b9db58-ecdf-4f8b-8149-9c701aed45f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:47.230842 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b9db58-ecdf-4f8b-8149-9c701aed45f4 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:47.231117 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b9db58-ecdf-4f8b-8149-9c701aed45f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:47.231384 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b9db58-ecdf-4f8b-8149-9c701aed45f4 None admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:47.232191 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b9db58-ecdf-4f8b-8149-9c701aed45f4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:47.232496 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b9db58-ecdf-4f8b-8149-9c701aed45f4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:47.233895 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b9db58-ecdf-4f8b-8149-9c701aed45f4 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:47.245249 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 539/1078] 10.222.0.58 () {56 vars in 1148 bytes} [Tue May 17 19:11:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:11:47.251137 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b9db58-ecdf-4f8b-8149-9c701aed45f4 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:47.255812 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f792a7f2-dd9b-4869-ab7b-ea65f11c0d17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:47.256602 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f792a7f2-dd9b-4869-ab7b-ea65f11c0d17 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:47.256882 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f792a7f2-dd9b-4869-ab7b-ea65f11c0d17 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:47.257156 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f792a7f2-dd9b-4869-ab7b-ea65f11c0d17 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:47.266992 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f792a7f2-dd9b-4869-ab7b-ea65f11c0d17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 17 19:11:47.274498 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-f792a7f2-dd9b-4869-ab7b-ea65f11c0d17 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 17 19:11:47.274498 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:47.274498 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:11:47.274498 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:11:47.274498 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:11:47.274498 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:11:47.274498 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:11:47.274498 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:11:47.274498 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:11:47.274498 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:11:47.274498 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:11:47.274498 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:11:47.274498 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 17 19:11:47.274498 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) May 17 19:11:47.274498 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 17 19:11:47.274498 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 17 19:11:47.275465 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 17 19:11:47.275465 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 17 19:11:47.275465 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 17 19:11:47.275465 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 17 19:11:47.275465 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:11:47.275465 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:11:47.275465 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:11:47.275465 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:11:47.275465 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:11:47.275465 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:11:47.275465 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:11:47.275465 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:11:47.275465 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:11:47.275465 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:11:47.275465 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:11:47.275465 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:11:47.276384 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:11:47.276384 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:11:47.276384 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:11:47.276384 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:11:47.276384 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 17 19:11:47.276384 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 17 19:11:47.276384 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 17 19:11:47.276384 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 17 19:11:47.276384 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 17 19:11:47.276384 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 17 19:11:47.276384 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 17 19:11:47.276384 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:11:47.276384 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 540/1079] 10.222.0.58 () {56 vars in 1096 bytes} [Tue May 17 19:11:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 17 19:11:47.281316 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f792a7f2-dd9b-4869-ab7b-ea65f11c0d17 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:47.346180 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b9db58-ecdf-4f8b-8149-9c701aed45f4 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:47.374737 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f792a7f2-dd9b-4869-ab7b-ea65f11c0d17 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:47.377348 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026d43b5-88d4-4bb3-a3d4-dff9a0619cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:47.378215 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026d43b5-88d4-4bb3-a3d4-dff9a0619cc2 None admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:47.378491 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026d43b5-88d4-4bb3-a3d4-dff9a0619cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:47.378760 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026d43b5-88d4-4bb3-a3d4-dff9a0619cc2 None admin] PATH_INFO: `/v3/projects/1a9d3a0631284c9fb113266036993e2d/users/65c1f878b0dc4d63a4f5dcb12fac1dc1/roles/897862af1c83498f86d0dfdf48bea276` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:47.381124 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026d43b5-88d4-4bb3-a3d4-dff9a0619cc2 None admin] RBAC: Authorizing `identity:create_grant(project_id=1a9d3a0631284c9fb113266036993e2d, user_id=65c1f878b0dc4d63a4f5dcb12fac1dc1, role_id=897862af1c83498f86d0dfdf48bea276)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:47.382543 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026d43b5-88d4-4bb3-a3d4-dff9a0619cc2 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:47.406141 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbbe06e-3008-49a5-a082-ad52385e7e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:47.406766 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbbe06e-3008-49a5-a082-ad52385e7e83 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:47.407566 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbbe06e-3008-49a5-a082-ad52385e7e83 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:47.407865 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbbe06e-3008-49a5-a082-ad52385e7e83 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:47.408595 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbbe06e-3008-49a5-a082-ad52385e7e83 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:47.408899 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbbe06e-3008-49a5-a082-ad52385e7e83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:47.410245 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbbe06e-3008-49a5-a082-ad52385e7e83 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:47.418490 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 541/1080] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 17 19:11:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:11:47.424918 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbbe06e-3008-49a5-a082-ad52385e7e83 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:47.503666 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 540/1081] 10.222.0.58 () {58 vars in 1439 bytes} [Tue May 17 19:11:47 2022] PUT /identity/v3/projects/1a9d3a0631284c9fb113266036993e2d/users/65c1f878b0dc4d63a4f5dcb12fac1dc1/roles/897862af1c83498f86d0dfdf48bea276 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 17 19:11:47.519282 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbbe06e-3008-49a5-a082-ad52385e7e83 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:47.555644 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131a2553-124d-409a-bc16-86d75033fcdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:47.556526 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131a2553-124d-409a-bc16-86d75033fcdc None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:47.556806 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131a2553-124d-409a-bc16-86d75033fcdc None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:47.557081 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131a2553-124d-409a-bc16-86d75033fcdc None admin] PATH_INFO: `/v3/users/bc78d499848447e28a145b13c385fa63` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:47.558369 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131a2553-124d-409a-bc16-86d75033fcdc None admin] RBAC: Authorizing `identity:get_user(user_id=bc78d499848447e28a145b13c385fa63)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:47.559876 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131a2553-124d-409a-bc16-86d75033fcdc None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:47.561612 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 542/1082] 10.222.0.58 () {56 vars in 1177 bytes} [Tue May 17 19:11:47 2022] GET /identity/v3/users/bc78d499848447e28a145b13c385fa63 => generated 285 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:11:47.567708 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026d43b5-88d4-4bb3-a3d4-dff9a0619cc2 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:47.668565 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026d43b5-88d4-4bb3-a3d4-dff9a0619cc2 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:47.702612 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bdd8b5-7c56-454f-99e7-0a9df62ca24e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:47.703455 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bdd8b5-7c56-454f-99e7-0a9df62ca24e None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:47.703831 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bdd8b5-7c56-454f-99e7-0a9df62ca24e None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:47.704590 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bdd8b5-7c56-454f-99e7-0a9df62ca24e None admin] PATH_INFO: `/v3/projects/2ff0f1d671ce48bab664650af5095826` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:47.705743 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bdd8b5-7c56-454f-99e7-0a9df62ca24e None admin] RBAC: Authorizing `identity:get_project(project_id=2ff0f1d671ce48bab664650af5095826)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:47.707187 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bdd8b5-7c56-454f-99e7-0a9df62ca24e None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:47.708903 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 541/1083] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 17 19:11:47 2022] GET /identity/v3/projects/2ff0f1d671ce48bab664650af5095826 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:47.714792 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131a2553-124d-409a-bc16-86d75033fcdc None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:47.809348 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131a2553-124d-409a-bc16-86d75033fcdc None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:47.842197 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd080e58-0ba1-489c-81a3-54481f425291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:47.843074 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd080e58-0ba1-489c-81a3-54481f425291 None admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:47.843348 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd080e58-0ba1-489c-81a3-54481f425291 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:47.843653 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd080e58-0ba1-489c-81a3-54481f425291 None admin] PATH_INFO: `/v3/projects/2ff0f1d671ce48bab664650af5095826/users/bc78d499848447e28a145b13c385fa63/roles/60af5088793e4379b88821a86552a344` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:47.845885 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd080e58-0ba1-489c-81a3-54481f425291 None admin] RBAC: Authorizing `identity:create_grant(project_id=2ff0f1d671ce48bab664650af5095826, user_id=bc78d499848447e28a145b13c385fa63, role_id=60af5088793e4379b88821a86552a344)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:47.847302 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd080e58-0ba1-489c-81a3-54481f425291 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:47.921117 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 543/1084] 10.222.0.58 () {58 vars in 1439 bytes} [Tue May 17 19:11:47 2022] PUT /identity/v3/projects/2ff0f1d671ce48bab664650af5095826/users/bc78d499848447e28a145b13c385fa63/roles/60af5088793e4379b88821a86552a344 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:11:49.700883 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d347e1c-8b5a-4b22-b8c1-a3504051091f None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:49.701437 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d347e1c-8b5a-4b22-b8c1-a3504051091f None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:49.701437 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d347e1c-8b5a-4b22-b8c1-a3504051091f None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:49.702778 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 542/1085] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:11:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:11:49.708195 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d5288c-9021-41cc-a7d9-7f28afbf4113 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:49.708613 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d5288c-9021-41cc-a7d9-7f28afbf4113 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:49.708719 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d5288c-9021-41cc-a7d9-7f28afbf4113 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:49.725134 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-e8d5288c-9021-41cc-a7d9-7f28afbf4113 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:11:49.726298 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-e8d5288c-9021-41cc-a7d9-7f28afbf4113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:11:49.788521 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 544/1086] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:11:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 17 19:11:49.811741 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d347e1c-8b5a-4b22-b8c1-a3504051091f None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:49.914678 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d347e1c-8b5a-4b22-b8c1-a3504051091f None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:49.948174 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366b9ff8-b319-4797-9f5c-f41e74fd39e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:49.948983 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366b9ff8-b319-4797-9f5c-f41e74fd39e3 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:49.949270 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366b9ff8-b319-4797-9f5c-f41e74fd39e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:49.949539 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366b9ff8-b319-4797-9f5c-f41e74fd39e3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:49.959867 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366b9ff8-b319-4797-9f5c-f41e74fd39e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 17 19:11:49.967672 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-366b9ff8-b319-4797-9f5c-f41e74fd39e3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 17 19:11:49.967672 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:49.967672 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:11:49.967672 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:11:49.967672 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:11:49.967672 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:11:49.967672 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:11:49.967672 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:11:49.967672 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:11:49.967672 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:11:49.967672 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:11:49.967672 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:11:49.967672 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 17 19:11:49.967672 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) May 17 19:11:49.967672 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 17 19:11:49.967672 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 17 19:11:49.968645 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 17 19:11:49.968645 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 17 19:11:49.968645 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 17 19:11:49.968645 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 17 19:11:49.968645 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:11:49.968645 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:11:49.968645 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:11:49.968645 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:11:49.968645 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:11:49.968645 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:11:49.968645 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:11:49.968645 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:11:49.968645 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:11:49.968645 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:11:49.968645 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:11:49.968645 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:11:49.969517 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:11:49.969517 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:11:49.969517 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:11:49.969517 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:11:49.969517 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 17 19:11:49.969517 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 17 19:11:49.969517 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 17 19:11:49.969517 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 17 19:11:49.969517 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 17 19:11:49.969517 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 17 19:11:49.969517 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 17 19:11:49.969517 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:11:49.969517 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 543/1087] 10.222.0.58 () {56 vars in 1120 bytes} [Tue May 17 19:11:49 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:11:49.975288 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d5288c-9021-41cc-a7d9-7f28afbf4113 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:50.068857 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d5288c-9021-41cc-a7d9-7f28afbf4113 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:50.072113 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403db428-572b-41db-b60b-1f370f199556 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:50.072396 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403db428-572b-41db-b60b-1f370f199556 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:50.072672 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403db428-572b-41db-b60b-1f370f199556 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:50.074033 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 544/1088] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:11:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 17 19:11:50.078982 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb7cf3e-f7e2-403b-ba8e-1263540fbb64 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:50.079262 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb7cf3e-f7e2-403b-ba8e-1263540fbb64 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:50.079555 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb7cf3e-f7e2-403b-ba8e-1263540fbb64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:50.096040 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-4bb7cf3e-f7e2-403b-ba8e-1263540fbb64 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:11:50.097205 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-4bb7cf3e-f7e2-403b-ba8e-1263540fbb64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:11:50.101801 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c375b37-fd17-4fe9-a47d-beb11cf46e3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:50.102463 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c375b37-fd17-4fe9-a47d-beb11cf46e3d None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:50.102739 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c375b37-fd17-4fe9-a47d-beb11cf46e3d None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:50.103008 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c375b37-fd17-4fe9-a47d-beb11cf46e3d None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:50.103769 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c375b37-fd17-4fe9-a47d-beb11cf46e3d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:50.104071 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c375b37-fd17-4fe9-a47d-beb11cf46e3d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:50.105433 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c375b37-fd17-4fe9-a47d-beb11cf46e3d None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:50.113660 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 545/1089] 10.222.0.58 () {56 vars in 1115 bytes} [Tue May 17 19:11:49 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:50.119093 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c375b37-fd17-4fe9-a47d-beb11cf46e3d None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:50.160587 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 545/1090] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:11:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 17 19:11:50.183319 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb7cf3e-f7e2-403b-ba8e-1263540fbb64 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:50.210905 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c375b37-fd17-4fe9-a47d-beb11cf46e3d None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:50.242678 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8a750a-2474-45f5-b7f2-c1d3f0d85998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:50.243459 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8a750a-2474-45f5-b7f2-c1d3f0d85998 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:50.243795 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8a750a-2474-45f5-b7f2-c1d3f0d85998 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:50.244020 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8a750a-2474-45f5-b7f2-c1d3f0d85998 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:50.254579 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8a750a-2474-45f5-b7f2-c1d3f0d85998 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:50.256078 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8a750a-2474-45f5-b7f2-c1d3f0d85998 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:50.273043 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-1c8a750a-2474-45f5-b7f2-c1d3f0d85998 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:11:50.273043 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:50.273043 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:11:50.273043 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:11:50.273043 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:11:50.273043 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:11:50.273043 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:11:50.273043 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:11:50.273043 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:11:50.273043 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:11:50.273043 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:11:50.273043 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:11:50.273043 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 17 19:11:50.273043 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 17 19:11:50.273043 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 17 19:11:50.273926 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 17 19:11:50.273926 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:11:50.273926 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:11:50.273926 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:11:50.273926 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:11:50.273926 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:11:50.273926 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:11:50.273926 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:11:50.273926 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:11:50.273926 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:11:50.273926 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:11:50.273926 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:11:50.273926 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:11:50.273926 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:11:50.273926 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:11:50.273926 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:11:50.273926 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:11:50.274842 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 17 19:11:50.274842 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 17 19:11:50.274842 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:11:50.274842 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:11:50.274842 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 546/1091] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:11:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 17 19:11:50.275622 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb7cf3e-f7e2-403b-ba8e-1263540fbb64 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:50.279813 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8a750a-2474-45f5-b7f2-c1d3f0d85998 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:50.307237 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615e5eea-682d-4b5e-86a3-b88b248cd9e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:50.308049 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615e5eea-682d-4b5e-86a3-b88b248cd9e3 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:50.308328 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615e5eea-682d-4b5e-86a3-b88b248cd9e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:50.308618 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615e5eea-682d-4b5e-86a3-b88b248cd9e3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:50.318647 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615e5eea-682d-4b5e-86a3-b88b248cd9e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 17 19:11:50.326079 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-615e5eea-682d-4b5e-86a3-b88b248cd9e3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 17 19:11:50.326079 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:50.326079 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:11:50.326079 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:11:50.326079 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:11:50.326079 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:11:50.326079 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:11:50.326079 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:11:50.326079 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:11:50.326079 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:11:50.326079 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:11:50.326079 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:11:50.326079 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 17 19:11:50.326079 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) May 17 19:11:50.326079 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 17 19:11:50.326079 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 17 19:11:50.327237 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 17 19:11:50.327237 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 17 19:11:50.327237 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 17 19:11:50.327237 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 17 19:11:50.327237 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:11:50.327237 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:11:50.327237 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:11:50.327237 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:11:50.327237 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:11:50.327237 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:11:50.327237 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:11:50.327237 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:11:50.327237 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:11:50.327237 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:11:50.327237 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:11:50.327237 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:11:50.328351 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:11:50.328351 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:11:50.328351 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:11:50.328351 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:11:50.328351 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 17 19:11:50.328351 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 17 19:11:50.328351 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 17 19:11:50.328351 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 17 19:11:50.328351 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 17 19:11:50.328351 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 17 19:11:50.328351 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 17 19:11:50.328351 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:11:50.331634 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 546/1092] 10.222.0.58 () {56 vars in 1096 bytes} [Tue May 17 19:11:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 17 19:11:50.333677 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615e5eea-682d-4b5e-86a3-b88b248cd9e3 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:50.373356 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8a750a-2474-45f5-b7f2-c1d3f0d85998 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:50.405866 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4651af-92e7-43ff-86b9-9a12b0089616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:50.406511 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4651af-92e7-43ff-86b9-9a12b0089616 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:50.406785 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4651af-92e7-43ff-86b9-9a12b0089616 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:50.407052 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4651af-92e7-43ff-86b9-9a12b0089616 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:50.407795 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4651af-92e7-43ff-86b9-9a12b0089616 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:50.408093 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4651af-92e7-43ff-86b9-9a12b0089616 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:50.409466 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4651af-92e7-43ff-86b9-9a12b0089616 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:50.420685 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 547/1093] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 17 19:11:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:11:50.426037 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615e5eea-682d-4b5e-86a3-b88b248cd9e3 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:50.426196 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4651af-92e7-43ff-86b9-9a12b0089616 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:50.462611 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a76967-38c3-436d-a142-7225bd1a45c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:50.463196 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a76967-38c3-436d-a142-7225bd1a45c0 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:50.463473 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a76967-38c3-436d-a142-7225bd1a45c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:50.463799 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a76967-38c3-436d-a142-7225bd1a45c0 None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:50.464563 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a76967-38c3-436d-a142-7225bd1a45c0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:50.464868 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a76967-38c3-436d-a142-7225bd1a45c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:50.466219 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a76967-38c3-436d-a142-7225bd1a45c0 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:50.474265 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 547/1094] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 17 19:11:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:11:50.479987 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a76967-38c3-436d-a142-7225bd1a45c0 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:50.522107 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4651af-92e7-43ff-86b9-9a12b0089616 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:50.554219 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a458259-8164-450d-88d3-e0bf7762726f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:50.554994 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a458259-8164-450d-88d3-e0bf7762726f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:50.555269 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a458259-8164-450d-88d3-e0bf7762726f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:50.555560 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a458259-8164-450d-88d3-e0bf7762726f None admin] PATH_INFO: `/v3/users/nova` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:50.562259 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a458259-8164-450d-88d3-e0bf7762726f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:50.564282 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a458259-8164-450d-88d3-e0bf7762726f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:50.574415 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a76967-38c3-436d-a142-7225bd1a45c0 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:50.578077 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-3a458259-8164-450d-88d3-e0bf7762726f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 17 19:11:50.578077 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:50.578077 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:11:50.578077 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:11:50.578077 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:11:50.578077 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:11:50.578077 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:11:50.578077 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:11:50.578077 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:11:50.578077 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:11:50.578077 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:11:50.578077 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:11:50.578077 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 17 19:11:50.578077 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_user(user_id) May 17 19:11:50.578077 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 17 19:11:50.578077 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 17 19:11:50.579057 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:11:50.579057 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:11:50.579057 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 17 19:11:50.579057 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 17 19:11:50.579057 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 17 19:11:50.579057 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 17 19:11:50.579057 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:11:50.579057 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:11:50.579057 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:11:50.579057 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:11:50.579057 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:11:50.579057 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:11:50.579057 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:11:50.579057 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:11:50.579057 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:11:50.579057 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:11:50.579057 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:11:50.579057 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:11:50.580133 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:11:50.580133 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:11:50.580133 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 17 19:11:50.580133 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 17 19:11:50.580133 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 17 19:11:50.580133 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 17 19:11:50.580133 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 17 19:11:50.580133 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 17 19:11:50.580133 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 17 19:11:50.580133 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:11:50.580133 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 548/1095] 10.222.0.58 () {56 vars in 1093 bytes} [Tue May 17 19:11:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 17 19:11:50.584834 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a458259-8164-450d-88d3-e0bf7762726f None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:50.606196 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ae5ae0-708e-4224-b9ca-540201f7f717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:50.607049 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ae5ae0-708e-4224-b9ca-540201f7f717 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:50.607325 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ae5ae0-708e-4224-b9ca-540201f7f717 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:50.607625 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ae5ae0-708e-4224-b9ca-540201f7f717 None admin] PATH_INFO: `/v3/users/bc78d499848447e28a145b13c385fa63` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:50.608833 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ae5ae0-708e-4224-b9ca-540201f7f717 None admin] RBAC: Authorizing `identity:get_user(user_id=bc78d499848447e28a145b13c385fa63)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:50.610799 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ae5ae0-708e-4224-b9ca-540201f7f717 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:50.612459 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 548/1096] 10.222.0.58 () {56 vars in 1177 bytes} [Tue May 17 19:11:50 2022] GET /identity/v3/users/bc78d499848447e28a145b13c385fa63 => generated 285 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:11:50.617963 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ae5ae0-708e-4224-b9ca-540201f7f717 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:50.677585 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a458259-8164-450d-88d3-e0bf7762726f None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:50.710432 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7d5766-cc0e-49c8-96e5-2989a060bcbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:50.711060 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ae5ae0-708e-4224-b9ca-540201f7f717 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:50.711427 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7d5766-cc0e-49c8-96e5-2989a060bcbb None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:50.711427 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7d5766-cc0e-49c8-96e5-2989a060bcbb None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:50.711648 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7d5766-cc0e-49c8-96e5-2989a060bcbb None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:50.712475 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7d5766-cc0e-49c8-96e5-2989a060bcbb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:50.712828 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7d5766-cc0e-49c8-96e5-2989a060bcbb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:50.714199 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7d5766-cc0e-49c8-96e5-2989a060bcbb None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:50.730975 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 549/1097] 10.222.0.58 () {56 vars in 1133 bytes} [Tue May 17 19:11:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:11:50.737066 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7d5766-cc0e-49c8-96e5-2989a060bcbb None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:50.743406 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c665e495-96c0-41b6-a881-df92814ece7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:50.744084 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c665e495-96c0-41b6-a881-df92814ece7f None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:50.744361 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c665e495-96c0-41b6-a881-df92814ece7f None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:50.744630 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c665e495-96c0-41b6-a881-df92814ece7f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:50.808600 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c665e495-96c0-41b6-a881-df92814ece7f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:50.810038 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c665e495-96c0-41b6-a881-df92814ece7f None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:50.830109 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7d5766-cc0e-49c8-96e5-2989a060bcbb None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:50.865323 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2592e29d-6a2c-4c89-bc31-91448b08dfce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:50.865970 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2592e29d-6a2c-4c89-bc31-91448b08dfce None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:50.866249 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2592e29d-6a2c-4c89-bc31-91448b08dfce None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:50.866518 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2592e29d-6a2c-4c89-bc31-91448b08dfce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:50.905476 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 549/1098] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 17 19:11:50 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 17 19:11:50.911231 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c665e495-96c0-41b6-a881-df92814ece7f None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:50.931196 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2592e29d-6a2c-4c89-bc31-91448b08dfce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:50.932796 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2592e29d-6a2c-4c89-bc31-91448b08dfce None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:51.058239 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c665e495-96c0-41b6-a881-df92814ece7f None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:51.060004 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 550/1099] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 17 19:11:50 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 17 19:11:51.066118 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2592e29d-6a2c-4c89-bc31-91448b08dfce None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:51.091714 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b6f039-d1c2-4abe-96b2-149d8f92a18b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:51.092565 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b6f039-d1c2-4abe-96b2-149d8f92a18b None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:51.092842 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b6f039-d1c2-4abe-96b2-149d8f92a18b None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:51.093110 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b6f039-d1c2-4abe-96b2-149d8f92a18b None admin] PATH_INFO: `/v3/projects/2ff0f1d671ce48bab664650af5095826` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:51.094278 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b6f039-d1c2-4abe-96b2-149d8f92a18b None admin] RBAC: Authorizing `identity:get_project(project_id=2ff0f1d671ce48bab664650af5095826)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:51.096033 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b6f039-d1c2-4abe-96b2-149d8f92a18b None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:51.097726 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 550/1100] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 17 19:11:50 2022] GET /identity/v3/projects/2ff0f1d671ce48bab664650af5095826 => generated 311 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:11:51.103640 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b6f039-d1c2-4abe-96b2-149d8f92a18b None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:51.159371 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2592e29d-6a2c-4c89-bc31-91448b08dfce None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:51.191996 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a447b0c7-c8eb-443e-b9ff-eea0e9e371ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:51.192768 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a447b0c7-c8eb-443e-b9ff-eea0e9e371ac None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:51.193037 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a447b0c7-c8eb-443e-b9ff-eea0e9e371ac None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:51.193301 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a447b0c7-c8eb-443e-b9ff-eea0e9e371ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:51.196770 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b6f039-d1c2-4abe-96b2-149d8f92a18b None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:51.202958 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a447b0c7-c8eb-443e-b9ff-eea0e9e371ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:51.204589 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a447b0c7-c8eb-443e-b9ff-eea0e9e371ac None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:51.221495 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-a447b0c7-c8eb-443e-b9ff-eea0e9e371ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:11:51.221495 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:51.221495 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:11:51.221495 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:11:51.221495 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:11:51.221495 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:11:51.221495 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:11:51.221495 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:11:51.221495 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:11:51.221495 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:11:51.221495 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:11:51.221495 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:11:51.221495 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 17 19:11:51.221495 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 17 19:11:51.221495 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 17 19:11:51.222401 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 17 19:11:51.222401 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:11:51.222401 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:11:51.222401 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:11:51.222401 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:11:51.222401 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:11:51.222401 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:11:51.222401 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:11:51.222401 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:11:51.222401 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:11:51.222401 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:11:51.222401 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:11:51.222401 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:11:51.222401 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:11:51.222401 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:11:51.222401 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:11:51.222401 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:11:51.223303 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 17 19:11:51.223303 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 17 19:11:51.223303 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:11:51.223303 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:11:51.223303 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 551/1101] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:11:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 17 19:11:51.228114 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a447b0c7-c8eb-443e-b9ff-eea0e9e371ac None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:51.228702 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398db45d-821d-4d84-8a23-5e5f5dafdbaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:51.229347 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398db45d-821d-4d84-8a23-5e5f5dafdbaa None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:51.229626 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398db45d-821d-4d84-8a23-5e5f5dafdbaa None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:51.229898 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398db45d-821d-4d84-8a23-5e5f5dafdbaa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:51.230725 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398db45d-821d-4d84-8a23-5e5f5dafdbaa None admin] RBAC: Adding query filter params (role.id=60af5088793e4379b88821a86552a344, scope.project.id=2ff0f1d671ce48bab664650af5095826, user.id=bc78d499848447e28a145b13c385fa63) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:51.231031 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398db45d-821d-4d84-8a23-5e5f5dafdbaa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:51.232467 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398db45d-821d-4d84-8a23-5e5f5dafdbaa None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:51.252438 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 551/1102] 10.222.0.58 () {56 vars in 1374 bytes} [Tue May 17 19:11:51 2022] GET /identity/v3/role_assignments?user.id=bc78d499848447e28a145b13c385fa63&scope.project.id=2ff0f1d671ce48bab664650af5095826&role.id=60af5088793e4379b88821a86552a344 => generated 608 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:11:51.328039 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a447b0c7-c8eb-443e-b9ff-eea0e9e371ac None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:51.363288 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16eed10-860c-41bc-bd88-e095a9f021cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:51.363991 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16eed10-860c-41bc-bd88-e095a9f021cd None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:51.364268 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16eed10-860c-41bc-bd88-e095a9f021cd None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:51.364535 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16eed10-860c-41bc-bd88-e095a9f021cd None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:51.365259 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16eed10-860c-41bc-bd88-e095a9f021cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:51.365554 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16eed10-860c-41bc-bd88-e095a9f021cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:51.366938 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16eed10-860c-41bc-bd88-e095a9f021cd None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:51.379063 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 552/1103] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 17 19:11:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:11:51.384877 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398db45d-821d-4d84-8a23-5e5f5dafdbaa None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:51.483285 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398db45d-821d-4d84-8a23-5e5f5dafdbaa None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:51.514572 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2f92dd-311b-426f-812c-07a8d00c6f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:51.515349 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2f92dd-311b-426f-812c-07a8d00c6f1f None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:51.515658 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2f92dd-311b-426f-812c-07a8d00c6f1f None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:51.515932 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2f92dd-311b-426f-812c-07a8d00c6f1f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:51.522771 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2f92dd-311b-426f-812c-07a8d00c6f1f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:51.524227 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2f92dd-311b-426f-812c-07a8d00c6f1f None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:51.538629 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-9b2f92dd-311b-426f-812c-07a8d00c6f1f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 17 19:11:51.538629 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:11:51.538629 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:11:51.538629 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:11:51.538629 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:11:51.538629 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:11:51.538629 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:11:51.538629 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:11:51.538629 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:11:51.538629 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:11:51.538629 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:11:51.538629 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:11:51.538629 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 17 19:11:51.538629 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(project_id) May 17 19:11:51.538629 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 17 19:11:51.539601 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 17 19:11:51.539601 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:11:51.539601 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:11:51.539601 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:11:51.539601 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:11:51.539601 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:11:51.539601 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:11:51.539601 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:11:51.539601 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:11:51.539601 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:11:51.539601 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:11:51.539601 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:11:51.539601 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:11:51.539601 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:11:51.539601 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:11:51.539601 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:11:51.539601 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:11:51.540594 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 17 19:11:51.540594 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 17 19:11:51.540594 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 17 19:11:51.540594 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 17 19:11:51.540594 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 17 19:11:51.540594 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 17 19:11:51.540594 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 17 19:11:51.540594 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:11:51.543782 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 552/1104] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 17 19:11:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:11:51.545581 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16eed10-860c-41bc-bd88-e095a9f021cd None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:51.642012 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16eed10-860c-41bc-bd88-e095a9f021cd None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:51.673538 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a00e36-f310-4bce-8db4-89a26a979ac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:51.674271 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a00e36-f310-4bce-8db4-89a26a979ac8 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:51.674554 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a00e36-f310-4bce-8db4-89a26a979ac8 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:51.674825 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a00e36-f310-4bce-8db4-89a26a979ac8 None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:51.675641 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a00e36-f310-4bce-8db4-89a26a979ac8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:51.675950 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a00e36-f310-4bce-8db4-89a26a979ac8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:51.677374 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a00e36-f310-4bce-8db4-89a26a979ac8 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:51.689359 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 553/1105] 10.222.0.58 () {56 vars in 1148 bytes} [Tue May 17 19:11:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:51.695280 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2f92dd-311b-426f-812c-07a8d00c6f1f None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:51.787873 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2f92dd-311b-426f-812c-07a8d00c6f1f None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:51.819478 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b052ac-ef47-433c-8176-7e7b9609d624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:51.820138 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b052ac-ef47-433c-8176-7e7b9609d624 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:51.820409 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b052ac-ef47-433c-8176-7e7b9609d624 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:51.820676 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b052ac-ef47-433c-8176-7e7b9609d624 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:51.821488 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b052ac-ef47-433c-8176-7e7b9609d624 None admin] RBAC: Adding query filter params (role.id=897862af1c83498f86d0dfdf48bea276, scope.project.id=1a9d3a0631284c9fb113266036993e2d, user.id=65c1f878b0dc4d63a4f5dcb12fac1dc1) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:51.821789 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b052ac-ef47-433c-8176-7e7b9609d624 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:51.823195 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b052ac-ef47-433c-8176-7e7b9609d624 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:51.828823 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 553/1106] 10.222.0.58 () {56 vars in 1374 bytes} [Tue May 17 19:11:51 2022] GET /identity/v3/role_assignments?user.id=65c1f878b0dc4d63a4f5dcb12fac1dc1&scope.project.id=1a9d3a0631284c9fb113266036993e2d&role.id=897862af1c83498f86d0dfdf48bea276 => generated 608 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:53.465942 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f19daea-6bfc-4e9f-8ff8-a925bf386e06 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:53.466468 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f19daea-6bfc-4e9f-8ff8-a925bf386e06 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:53.466468 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f19daea-6bfc-4e9f-8ff8-a925bf386e06 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:53.467860 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 554/1107] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:11:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:11:53.473365 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934b0562-c69a-441d-b3a2-92a7031f84c9 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:53.473780 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934b0562-c69a-441d-b3a2-92a7031f84c9 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:53.473931 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934b0562-c69a-441d-b3a2-92a7031f84c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:53.490481 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-934b0562-c69a-441d-b3a2-92a7031f84c9 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:11:53.491673 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-934b0562-c69a-441d-b3a2-92a7031f84c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:11:53.553535 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 554/1108] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:11:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 17 19:11:53.577473 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f19daea-6bfc-4e9f-8ff8-a925bf386e06 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:53.673356 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f19daea-6bfc-4e9f-8ff8-a925bf386e06 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:53.704717 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9afbf11-0adc-4f64-a8da-936fe702a9f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:53.705480 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9afbf11-0adc-4f64-a8da-936fe702a9f1 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:53.705752 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9afbf11-0adc-4f64-a8da-936fe702a9f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:53.706017 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9afbf11-0adc-4f64-a8da-936fe702a9f1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:53.707144 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9afbf11-0adc-4f64-a8da-936fe702a9f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:53.708655 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9afbf11-0adc-4f64-a8da-936fe702a9f1 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:53.710228 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 555/1109] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:11:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:53.715901 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934b0562-c69a-441d-b3a2-92a7031f84c9 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:53.806910 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934b0562-c69a-441d-b3a2-92a7031f84c9 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:53.838222 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6f4027-ccac-4785-875c-c08d3af98c1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:53.838863 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6f4027-ccac-4785-875c-c08d3af98c1e None admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:53.839141 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6f4027-ccac-4785-875c-c08d3af98c1e None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:53.839412 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6f4027-ccac-4785-875c-c08d3af98c1e None admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:53.840137 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6f4027-ccac-4785-875c-c08d3af98c1e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:53.841536 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6f4027-ccac-4785-875c-c08d3af98c1e None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:53.938184 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 555/1110] 10.222.0.58 () {60 vars in 1132 bytes} [Tue May 17 19:11:53 2022] POST /identity/v3/users => generated 286 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:11:56.154797 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caf2dd7-ecc5-4032-b89f-c590d010286a None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:56.155321 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caf2dd7-ecc5-4032-b89f-c590d010286a None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:56.155321 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caf2dd7-ecc5-4032-b89f-c590d010286a None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:56.157252 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 556/1111] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:11:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:11:56.162398 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdde93f-c84c-4b5f-9a65-5d4fd4c15713 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:56.162812 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdde93f-c84c-4b5f-9a65-5d4fd4c15713 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:56.162949 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdde93f-c84c-4b5f-9a65-5d4fd4c15713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:56.179554 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-dbdde93f-c84c-4b5f-9a65-5d4fd4c15713 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:11:56.180708 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-dbdde93f-c84c-4b5f-9a65-5d4fd4c15713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:11:56.241789 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 556/1112] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:11:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 17 19:11:56.264110 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caf2dd7-ecc5-4032-b89f-c590d010286a None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:56.358781 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caf2dd7-ecc5-4032-b89f-c590d010286a None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:56.391727 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c08148-f931-44b9-9760-c1be9e89fec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:56.392547 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c08148-f931-44b9-9760-c1be9e89fec2 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:56.392824 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c08148-f931-44b9-9760-c1be9e89fec2 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:56.393096 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c08148-f931-44b9-9760-c1be9e89fec2 None admin] PATH_INFO: `/v3/roles/1055a3235b0143e58f1090c1e154e5c1` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:56.394531 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c08148-f931-44b9-9760-c1be9e89fec2 None admin] RBAC: Authorizing `identity:get_role(role_id=1055a3235b0143e58f1090c1e154e5c1)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:56.395921 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c08148-f931-44b9-9760-c1be9e89fec2 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:56.397254 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 557/1113] 10.222.0.58 () {56 vars in 1177 bytes} [Tue May 17 19:11:56 2022] GET /identity/v3/roles/1055a3235b0143e58f1090c1e154e5c1 => generated 223 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:56.402770 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdde93f-c84c-4b5f-9a65-5d4fd4c15713 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:56.494054 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdde93f-c84c-4b5f-9a65-5d4fd4c15713 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:56.524907 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f853acc-1146-42c6-857d-788dd8a637b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:56.525750 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f853acc-1146-42c6-857d-788dd8a637b5 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:56.526023 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f853acc-1146-42c6-857d-788dd8a637b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:56.526288 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f853acc-1146-42c6-857d-788dd8a637b5 None admin] PATH_INFO: `/v3/users/48e5efdf513b4d449134bbc62487ab2b` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:56.539154 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f853acc-1146-42c6-857d-788dd8a637b5 None admin] RBAC: Authorizing `identity:get_user(user_id=48e5efdf513b4d449134bbc62487ab2b)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:56.540588 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f853acc-1146-42c6-857d-788dd8a637b5 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:56.542175 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 557/1114] 10.222.0.58 () {56 vars in 1177 bytes} [Tue May 17 19:11:56 2022] GET /identity/v3/users/48e5efdf513b4d449134bbc62487ab2b => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:56.548159 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c08148-f931-44b9-9760-c1be9e89fec2 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:56.641391 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c08148-f931-44b9-9760-c1be9e89fec2 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:56.672963 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f6b49d-104d-400a-b30e-c070d06ea8de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:56.673583 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f6b49d-104d-400a-b30e-c070d06ea8de None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:56.673855 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f6b49d-104d-400a-b30e-c070d06ea8de None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:56.674122 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f6b49d-104d-400a-b30e-c070d06ea8de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:56.736371 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f6b49d-104d-400a-b30e-c070d06ea8de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:56.737788 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f6b49d-104d-400a-b30e-c070d06ea8de None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:56.863918 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 558/1115] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 17 19:11:56 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 17 19:11:56.869553 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f853acc-1146-42c6-857d-788dd8a637b5 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:56.961157 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f853acc-1146-42c6-857d-788dd8a637b5 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:56.993517 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa85b77-ab47-46c9-8b87-e61357e62c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:56.994356 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa85b77-ab47-46c9-8b87-e61357e62c07 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:56.994630 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa85b77-ab47-46c9-8b87-e61357e62c07 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:56.994898 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa85b77-ab47-46c9-8b87-e61357e62c07 None admin] PATH_INFO: `/v3/projects/2ff0f1d671ce48bab664650af5095826` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:56.996064 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa85b77-ab47-46c9-8b87-e61357e62c07 None admin] RBAC: Authorizing `identity:get_project(project_id=2ff0f1d671ce48bab664650af5095826)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:56.997471 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa85b77-ab47-46c9-8b87-e61357e62c07 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:56.999129 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 558/1116] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 17 19:11:56 2022] GET /identity/v3/projects/2ff0f1d671ce48bab664650af5095826 => generated 311 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:57.005261 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f6b49d-104d-400a-b30e-c070d06ea8de None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:57.098654 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f6b49d-104d-400a-b30e-c070d06ea8de None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:57.130562 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a6eccc-3e6e-4c27-8a22-ca718f4e7257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:57.131132 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a6eccc-3e6e-4c27-8a22-ca718f4e7257 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:57.131408 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a6eccc-3e6e-4c27-8a22-ca718f4e7257 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:57.131705 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a6eccc-3e6e-4c27-8a22-ca718f4e7257 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:57.132519 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a6eccc-3e6e-4c27-8a22-ca718f4e7257 None admin] RBAC: Adding query filter params (role.id=1055a3235b0143e58f1090c1e154e5c1, scope.project.id=2ff0f1d671ce48bab664650af5095826, user.id=48e5efdf513b4d449134bbc62487ab2b) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:11:57.132823 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a6eccc-3e6e-4c27-8a22-ca718f4e7257 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:57.134233 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a6eccc-3e6e-4c27-8a22-ca718f4e7257 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:57.139395 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 559/1117] 10.222.0.58 () {56 vars in 1374 bytes} [Tue May 17 19:11:57 2022] GET /identity/v3/role_assignments?user.id=48e5efdf513b4d449134bbc62487ab2b&scope.project.id=2ff0f1d671ce48bab664650af5095826&role.id=1055a3235b0143e58f1090c1e154e5c1 => generated 259 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:59.301865 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0be61f-b980-4f1a-a70f-f72b84aa7c42 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:59.302367 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0be61f-b980-4f1a-a70f-f72b84aa7c42 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:59.302367 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0be61f-b980-4f1a-a70f-f72b84aa7c42 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:59.303727 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 559/1118] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:11:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:11:59.309012 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c39344d-2dc8-4ea9-bde8-02201eb98c86 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:59.309384 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c39344d-2dc8-4ea9-bde8-02201eb98c86 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:59.309566 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c39344d-2dc8-4ea9-bde8-02201eb98c86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:59.326218 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-8c39344d-2dc8-4ea9-bde8-02201eb98c86 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:11:59.327394 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-8c39344d-2dc8-4ea9-bde8-02201eb98c86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:11:59.389252 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 560/1119] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:11:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 17 19:11:59.411944 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0be61f-b980-4f1a-a70f-f72b84aa7c42 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:59.505794 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0be61f-b980-4f1a-a70f-f72b84aa7c42 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:59.536589 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b734a7c-f09c-409f-aeeb-9904f76ccdf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:59.537416 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b734a7c-f09c-409f-aeeb-9904f76ccdf7 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:59.537690 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b734a7c-f09c-409f-aeeb-9904f76ccdf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:59.537958 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b734a7c-f09c-409f-aeeb-9904f76ccdf7 None admin] PATH_INFO: `/v3/roles/1055a3235b0143e58f1090c1e154e5c1` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:59.539400 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b734a7c-f09c-409f-aeeb-9904f76ccdf7 None admin] RBAC: Authorizing `identity:get_role(role_id=1055a3235b0143e58f1090c1e154e5c1)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:59.540774 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b734a7c-f09c-409f-aeeb-9904f76ccdf7 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:59.542104 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 560/1120] 10.222.0.58 () {56 vars in 1177 bytes} [Tue May 17 19:11:59 2022] GET /identity/v3/roles/1055a3235b0143e58f1090c1e154e5c1 => generated 223 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:59.548245 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c39344d-2dc8-4ea9-bde8-02201eb98c86 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:59.639537 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c39344d-2dc8-4ea9-bde8-02201eb98c86 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:59.671264 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2d9873-2ec4-4c6e-8407-68e059b9c9ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:59.672668 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2d9873-2ec4-4c6e-8407-68e059b9c9ad None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:59.672940 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2d9873-2ec4-4c6e-8407-68e059b9c9ad None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:59.673203 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2d9873-2ec4-4c6e-8407-68e059b9c9ad None admin] PATH_INFO: `/v3/users/48e5efdf513b4d449134bbc62487ab2b` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:59.674363 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2d9873-2ec4-4c6e-8407-68e059b9c9ad None admin] RBAC: Authorizing `identity:get_user(user_id=48e5efdf513b4d449134bbc62487ab2b)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:59.675802 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2d9873-2ec4-4c6e-8407-68e059b9c9ad None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:59.677469 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 561/1121] 10.222.0.58 () {56 vars in 1177 bytes} [Tue May 17 19:11:59 2022] GET /identity/v3/users/48e5efdf513b4d449134bbc62487ab2b => generated 286 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:59.683124 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b734a7c-f09c-409f-aeeb-9904f76ccdf7 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:59.775977 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b734a7c-f09c-409f-aeeb-9904f76ccdf7 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:59.807360 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52d717d-c798-406b-9aba-119781ae95de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:59.808248 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52d717d-c798-406b-9aba-119781ae95de None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:59.808526 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52d717d-c798-406b-9aba-119781ae95de None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:59.808795 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52d717d-c798-406b-9aba-119781ae95de None admin] PATH_INFO: `/v3/projects/2ff0f1d671ce48bab664650af5095826` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:59.809938 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52d717d-c798-406b-9aba-119781ae95de None admin] RBAC: Authorizing `identity:get_project(project_id=2ff0f1d671ce48bab664650af5095826)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:59.811351 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52d717d-c798-406b-9aba-119781ae95de None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:11:59.813107 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 561/1122] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 17 19:11:59 2022] GET /identity/v3/projects/2ff0f1d671ce48bab664650af5095826 => generated 311 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:11:59.818671 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2d9873-2ec4-4c6e-8407-68e059b9c9ad None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:11:59.912785 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2d9873-2ec4-4c6e-8407-68e059b9c9ad None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:11:59.944342 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fb34ff-b581-4fcf-ab87-d72b8ef2fadc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:11:59.945209 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fb34ff-b581-4fcf-ab87-d72b8ef2fadc None admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:11:59.945486 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fb34ff-b581-4fcf-ab87-d72b8ef2fadc None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:11:59.945753 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fb34ff-b581-4fcf-ab87-d72b8ef2fadc None admin] PATH_INFO: `/v3/projects/2ff0f1d671ce48bab664650af5095826/users/48e5efdf513b4d449134bbc62487ab2b/roles/1055a3235b0143e58f1090c1e154e5c1` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:11:59.948092 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fb34ff-b581-4fcf-ab87-d72b8ef2fadc None admin] RBAC: Authorizing `identity:create_grant(project_id=2ff0f1d671ce48bab664650af5095826, user_id=48e5efdf513b4d449134bbc62487ab2b, role_id=1055a3235b0143e58f1090c1e154e5c1)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:11:59.949522 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fb34ff-b581-4fcf-ab87-d72b8ef2fadc None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:12:00.013974 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 562/1123] 10.222.0.58 () {58 vars in 1439 bytes} [Tue May 17 19:11:59 2022] PUT /identity/v3/projects/2ff0f1d671ce48bab664650af5095826/users/48e5efdf513b4d449134bbc62487ab2b/roles/1055a3235b0143e58f1090c1e154e5c1 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:12:02.161202 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d5944e-e0e9-4323-b2b5-410f2168be5b None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:12:02.161740 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d5944e-e0e9-4323-b2b5-410f2168be5b None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:12:02.161740 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d5944e-e0e9-4323-b2b5-410f2168be5b None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:12:02.163131 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 562/1124] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:12:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:12:02.168373 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459f34b5-029b-4e9e-babb-fe429dba7a8c None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:12:02.168811 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459f34b5-029b-4e9e-babb-fe429dba7a8c None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:12:02.168905 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459f34b5-029b-4e9e-babb-fe429dba7a8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:12:02.185398 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-459f34b5-029b-4e9e-babb-fe429dba7a8c None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:12:02.186566 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-459f34b5-029b-4e9e-babb-fe429dba7a8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:12:02.248411 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 563/1125] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:12:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 17 19:12:02.270502 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d5944e-e0e9-4323-b2b5-410f2168be5b None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:12:02.364651 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d5944e-e0e9-4323-b2b5-410f2168be5b None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:12:02.395371 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac43cc8-01c5-4157-a732-bdcfd95f5717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:12:02.396224 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac43cc8-01c5-4157-a732-bdcfd95f5717 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:12:02.396494 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac43cc8-01c5-4157-a732-bdcfd95f5717 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:12:02.396756 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac43cc8-01c5-4157-a732-bdcfd95f5717 None admin] PATH_INFO: `/v3/roles/1055a3235b0143e58f1090c1e154e5c1` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:12:02.398184 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac43cc8-01c5-4157-a732-bdcfd95f5717 None admin] RBAC: Authorizing `identity:get_role(role_id=1055a3235b0143e58f1090c1e154e5c1)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:12:02.399557 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac43cc8-01c5-4157-a732-bdcfd95f5717 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:12:02.400891 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 563/1126] 10.222.0.58 () {56 vars in 1177 bytes} [Tue May 17 19:12:02 2022] GET /identity/v3/roles/1055a3235b0143e58f1090c1e154e5c1 => generated 223 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:12:02.407203 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459f34b5-029b-4e9e-babb-fe429dba7a8c None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:12:02.499358 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459f34b5-029b-4e9e-babb-fe429dba7a8c None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:12:02.530503 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea35157-dac7-4df8-88c7-8f8574eaec0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:12:02.531947 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea35157-dac7-4df8-88c7-8f8574eaec0f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:12:02.532220 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea35157-dac7-4df8-88c7-8f8574eaec0f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:12:02.532483 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea35157-dac7-4df8-88c7-8f8574eaec0f None admin] PATH_INFO: `/v3/users/48e5efdf513b4d449134bbc62487ab2b` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:12:02.533631 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea35157-dac7-4df8-88c7-8f8574eaec0f None admin] RBAC: Authorizing `identity:get_user(user_id=48e5efdf513b4d449134bbc62487ab2b)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:12:02.535038 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea35157-dac7-4df8-88c7-8f8574eaec0f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:12:02.536645 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 564/1127] 10.222.0.58 () {56 vars in 1177 bytes} [Tue May 17 19:12:02 2022] GET /identity/v3/users/48e5efdf513b4d449134bbc62487ab2b => generated 286 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:12:02.541988 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac43cc8-01c5-4157-a732-bdcfd95f5717 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:12:02.633793 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac43cc8-01c5-4157-a732-bdcfd95f5717 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:12:02.665843 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2e66e4-02a3-41fc-a00e-5b7dd7b66a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:12:02.666457 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2e66e4-02a3-41fc-a00e-5b7dd7b66a78 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:12:02.666730 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2e66e4-02a3-41fc-a00e-5b7dd7b66a78 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:12:02.666996 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2e66e4-02a3-41fc-a00e-5b7dd7b66a78 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:12:02.729443 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2e66e4-02a3-41fc-a00e-5b7dd7b66a78 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:12:02.730892 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2e66e4-02a3-41fc-a00e-5b7dd7b66a78 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:12:02.820916 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 564/1128] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 17 19:12:02 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 17 19:12:02.826644 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea35157-dac7-4df8-88c7-8f8574eaec0f None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:12:02.920456 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea35157-dac7-4df8-88c7-8f8574eaec0f None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:12:02.952348 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b89352-7583-4e19-9ee6-8b14e6b22184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:12:02.953195 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b89352-7583-4e19-9ee6-8b14e6b22184 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:12:02.953472 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b89352-7583-4e19-9ee6-8b14e6b22184 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:12:02.953747 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b89352-7583-4e19-9ee6-8b14e6b22184 None admin] PATH_INFO: `/v3/projects/2ff0f1d671ce48bab664650af5095826` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:12:02.954897 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b89352-7583-4e19-9ee6-8b14e6b22184 None admin] RBAC: Authorizing `identity:get_project(project_id=2ff0f1d671ce48bab664650af5095826)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:12:02.956407 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b89352-7583-4e19-9ee6-8b14e6b22184 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:12:02.958083 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 565/1129] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 17 19:12:02 2022] GET /identity/v3/projects/2ff0f1d671ce48bab664650af5095826 => generated 311 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:12:02.964073 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2e66e4-02a3-41fc-a00e-5b7dd7b66a78 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:12:03.056644 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2e66e4-02a3-41fc-a00e-5b7dd7b66a78 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:12:03.089347 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7332c592-3b5c-430a-81c7-b83d5e0b6c2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:12:03.089934 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7332c592-3b5c-430a-81c7-b83d5e0b6c2b None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:12:03.090218 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7332c592-3b5c-430a-81c7-b83d5e0b6c2b None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:12:03.090494 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7332c592-3b5c-430a-81c7-b83d5e0b6c2b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:12:03.091317 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7332c592-3b5c-430a-81c7-b83d5e0b6c2b None admin] RBAC: Adding query filter params (role.id=1055a3235b0143e58f1090c1e154e5c1, scope.project.id=2ff0f1d671ce48bab664650af5095826, user.id=48e5efdf513b4d449134bbc62487ab2b) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:12:03.091653 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7332c592-3b5c-430a-81c7-b83d5e0b6c2b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:12:03.093074 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7332c592-3b5c-430a-81c7-b83d5e0b6c2b None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:12:03.098898 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 565/1130] 10.222.0.58 () {56 vars in 1374 bytes} [Tue May 17 19:12:02 2022] GET /identity/v3/role_assignments?user.id=48e5efdf513b4d449134bbc62487ab2b&scope.project.id=2ff0f1d671ce48bab664650af5095826&role.id=1055a3235b0143e58f1090c1e154e5c1 => generated 608 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:12:05.293692 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22a27f2-cb44-4324-9966-122c7e02bd57 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:12:05.294263 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22a27f2-cb44-4324-9966-122c7e02bd57 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:12:05.294263 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22a27f2-cb44-4324-9966-122c7e02bd57 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:12:05.295550 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 566/1131] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:12:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:12:05.301338 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdaf276-1b21-454b-9ded-a2e68b7a3057 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:12:05.301758 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdaf276-1b21-454b-9ded-a2e68b7a3057 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:12:05.301868 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdaf276-1b21-454b-9ded-a2e68b7a3057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:12:05.317844 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-3bdaf276-1b21-454b-9ded-a2e68b7a3057 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:12:05.318997 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-3bdaf276-1b21-454b-9ded-a2e68b7a3057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:12:05.380192 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 566/1132] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:12:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 17 19:12:05.402454 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22a27f2-cb44-4324-9966-122c7e02bd57 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:12:05.498387 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22a27f2-cb44-4324-9966-122c7e02bd57 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:12:05.530045 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfbc735-7e78-47b2-a760-9b410ad08ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:12:05.530815 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfbc735-7e78-47b2-a760-9b410ad08ddf None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:12:05.531725 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfbc735-7e78-47b2-a760-9b410ad08ddf None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:12:05.531999 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfbc735-7e78-47b2-a760-9b410ad08ddf None admin] PATH_INFO: `/v3/domains/default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:12:05.533157 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfbc735-7e78-47b2-a760-9b410ad08ddf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:12:05.534578 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfbc735-7e78-47b2-a760-9b410ad08ddf None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:12:05.536191 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 567/1133] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:12:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:12:05.541807 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdaf276-1b21-454b-9ded-a2e68b7a3057 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:12:05.632673 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdaf276-1b21-454b-9ded-a2e68b7a3057 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:12:05.664278 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230d7e70-4645-4b2d-82d4-caf7c039a6f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:12:05.664912 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230d7e70-4645-4b2d-82d4-caf7c039a6f3 None admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:12:05.665186 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230d7e70-4645-4b2d-82d4-caf7c039a6f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:12:05.665453 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230d7e70-4645-4b2d-82d4-caf7c039a6f3 None admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:12:05.666136 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230d7e70-4645-4b2d-82d4-caf7c039a6f3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:12:05.667552 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230d7e70-4645-4b2d-82d4-caf7c039a6f3 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:12:05.752271 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 567/1134] 10.222.0.58 () {60 vars in 1141 bytes} [Tue May 17 19:12:05 2022] POST /identity/v3/projects => generated 311 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:12:07.943215 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47300da-6e26-4dba-a121-ef5f5c28fdb1 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:12:07.943798 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47300da-6e26-4dba-a121-ef5f5c28fdb1 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:12:07.943798 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47300da-6e26-4dba-a121-ef5f5c28fdb1 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:12:07.945162 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 568/1135] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:12:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:12:07.950610 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1594be8-f084-4e6c-90da-5e6c3fd5dd15 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:12:07.950981 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1594be8-f084-4e6c-90da-5e6c3fd5dd15 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:12:07.951162 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1594be8-f084-4e6c-90da-5e6c3fd5dd15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:12:07.967268 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-a1594be8-f084-4e6c-90da-5e6c3fd5dd15 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:12:07.968502 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-a1594be8-f084-4e6c-90da-5e6c3fd5dd15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:12:08.029916 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 568/1136] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:12:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 17 19:12:08.054170 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47300da-6e26-4dba-a121-ef5f5c28fdb1 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:12:08.148763 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47300da-6e26-4dba-a121-ef5f5c28fdb1 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:12:08.180566 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95811f5d-2b74-41a5-a993-db77fb80ad13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:12:08.181328 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95811f5d-2b74-41a5-a993-db77fb80ad13 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:12:08.181601 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95811f5d-2b74-41a5-a993-db77fb80ad13 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:12:08.181866 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95811f5d-2b74-41a5-a993-db77fb80ad13 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:12:08.183003 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95811f5d-2b74-41a5-a993-db77fb80ad13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:12:08.184461 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95811f5d-2b74-41a5-a993-db77fb80ad13 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:12:08.186036 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 569/1137] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:12:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:12:08.191634 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1594be8-f084-4e6c-90da-5e6c3fd5dd15 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:12:08.283812 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1594be8-f084-4e6c-90da-5e6c3fd5dd15 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:12:08.315624 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b102306-eeac-44b7-89d1-d87f5427643c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:12:08.316268 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b102306-eeac-44b7-89d1-d87f5427643c None admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:12:08.316547 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b102306-eeac-44b7-89d1-d87f5427643c None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:12:08.316819 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b102306-eeac-44b7-89d1-d87f5427643c None admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:12:08.317522 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b102306-eeac-44b7-89d1-d87f5427643c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:12:08.318931 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b102306-eeac-44b7-89d1-d87f5427643c None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:12:08.433702 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 569/1138] 10.222.0.58 () {60 vars in 1132 bytes} [Tue May 17 19:12:08 2022] POST /identity/v3/users => generated 286 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:12:10.635827 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da13fe8-120b-4861-a7e4-1b2181587bac None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:12:10.636376 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da13fe8-120b-4861-a7e4-1b2181587bac None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:12:10.636376 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da13fe8-120b-4861-a7e4-1b2181587bac None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:12:10.637715 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 570/1139] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:12:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:12:10.642975 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e5742d-15a3-4330-b8d0-91eb11c44de6 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:12:10.643397 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e5742d-15a3-4330-b8d0-91eb11c44de6 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:12:10.643527 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e5742d-15a3-4330-b8d0-91eb11c44de6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:12:10.660820 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-48e5742d-15a3-4330-b8d0-91eb11c44de6 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:12:10.661986 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-48e5742d-15a3-4330-b8d0-91eb11c44de6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:12:10.730967 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 570/1140] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:12:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 17 19:12:10.753567 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da13fe8-120b-4861-a7e4-1b2181587bac None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:12:10.849226 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da13fe8-120b-4861-a7e4-1b2181587bac None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:12:10.881088 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e06abc-02ce-4031-9179-b83e628f77ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:12:10.881863 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e06abc-02ce-4031-9179-b83e628f77ba None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:12:10.882142 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e06abc-02ce-4031-9179-b83e628f77ba None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:12:10.882414 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e06abc-02ce-4031-9179-b83e628f77ba None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:12:10.892814 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e06abc-02ce-4031-9179-b83e628f77ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 17 19:12:10.900257 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-03e06abc-02ce-4031-9179-b83e628f77ba None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 17 19:12:10.900257 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:12:10.900257 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:12:10.900257 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:12:10.900257 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:12:10.900257 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:12:10.900257 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:12:10.900257 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:12:10.900257 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:12:10.900257 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:12:10.900257 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:12:10.900257 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:12:10.900257 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 17 19:12:10.900257 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) May 17 19:12:10.900257 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 17 19:12:10.900257 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 17 19:12:10.901373 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 17 19:12:10.901373 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 17 19:12:10.901373 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 17 19:12:10.901373 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 17 19:12:10.901373 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:12:10.901373 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:12:10.901373 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:12:10.901373 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:12:10.901373 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:12:10.901373 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:12:10.901373 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:12:10.901373 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:12:10.901373 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:12:10.901373 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:12:10.901373 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:12:10.901373 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:12:10.907747 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:12:10.907747 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:12:10.907747 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:12:10.907747 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:12:10.907747 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 17 19:12:10.907747 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 17 19:12:10.907747 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 17 19:12:10.907747 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 17 19:12:10.907747 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 17 19:12:10.907747 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 17 19:12:10.907747 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 17 19:12:10.907747 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:12:10.907747 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 571/1141] 10.222.0.58 () {56 vars in 1096 bytes} [Tue May 17 19:12:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:12:10.908717 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e5742d-15a3-4330-b8d0-91eb11c44de6 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:12:10.998983 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e5742d-15a3-4330-b8d0-91eb11c44de6 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:12:11.030342 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2d0f12-44cf-4931-8bd9-e1dafb1af68a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:12:11.031590 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2d0f12-44cf-4931-8bd9-e1dafb1af68a None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:12:11.031867 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2d0f12-44cf-4931-8bd9-e1dafb1af68a None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:12:11.032134 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2d0f12-44cf-4931-8bd9-e1dafb1af68a None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:12:11.032921 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2d0f12-44cf-4931-8bd9-e1dafb1af68a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:12:11.033163 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2d0f12-44cf-4931-8bd9-e1dafb1af68a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:12:11.034507 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2d0f12-44cf-4931-8bd9-e1dafb1af68a None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:12:11.042044 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 571/1142] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 17 19:12:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:12:11.048302 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e06abc-02ce-4031-9179-b83e628f77ba None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:12:11.140842 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e06abc-02ce-4031-9179-b83e628f77ba None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:12:11.173163 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0596095f-3a94-48b5-ae03-45da83918f0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:12:11.174141 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0596095f-3a94-48b5-ae03-45da83918f0e None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:12:11.174415 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0596095f-3a94-48b5-ae03-45da83918f0e None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:12:11.174684 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0596095f-3a94-48b5-ae03-45da83918f0e None admin] PATH_INFO: `/v3/users/0580afe7e0d249bca0859ce781e6a66c` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:12:11.187664 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0596095f-3a94-48b5-ae03-45da83918f0e None admin] RBAC: Authorizing `identity:get_user(user_id=0580afe7e0d249bca0859ce781e6a66c)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:12:11.189098 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0596095f-3a94-48b5-ae03-45da83918f0e None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:12:11.190699 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 572/1143] 10.222.0.58 () {56 vars in 1177 bytes} [Tue May 17 19:12:11 2022] GET /identity/v3/users/0580afe7e0d249bca0859ce781e6a66c => generated 286 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:12:11.196414 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2d0f12-44cf-4931-8bd9-e1dafb1af68a None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:12:11.289858 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2d0f12-44cf-4931-8bd9-e1dafb1af68a None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:12:11.321181 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91caa13-4423-45dc-94a6-6e3d5e5bf68b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:12:11.321797 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91caa13-4423-45dc-94a6-6e3d5e5bf68b None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:12:11.322103 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91caa13-4423-45dc-94a6-6e3d5e5bf68b None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:12:11.322372 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91caa13-4423-45dc-94a6-6e3d5e5bf68b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:12:11.384658 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91caa13-4423-45dc-94a6-6e3d5e5bf68b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:12:11.386080 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91caa13-4423-45dc-94a6-6e3d5e5bf68b None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:12:11.476296 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 572/1144] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 17 19:12:11 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 17 19:12:11.481889 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0596095f-3a94-48b5-ae03-45da83918f0e None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:12:11.574217 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0596095f-3a94-48b5-ae03-45da83918f0e None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:12:11.605221 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f2e209-aca5-461e-a783-716d51cfee0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:12:11.606043 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f2e209-aca5-461e-a783-716d51cfee0f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:12:11.606317 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f2e209-aca5-461e-a783-716d51cfee0f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:12:11.606582 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f2e209-aca5-461e-a783-716d51cfee0f None admin] PATH_INFO: `/v3/projects/8e912761be6448d1a82682d8d1054a80` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:12:11.607744 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f2e209-aca5-461e-a783-716d51cfee0f None admin] RBAC: Authorizing `identity:get_project(project_id=8e912761be6448d1a82682d8d1054a80)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:12:11.609667 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f2e209-aca5-461e-a783-716d51cfee0f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:12:11.611290 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 573/1145] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 17 19:12:11 2022] GET /identity/v3/projects/8e912761be6448d1a82682d8d1054a80 => generated 311 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:12:11.617060 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91caa13-4423-45dc-94a6-6e3d5e5bf68b None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:12:11.711506 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91caa13-4423-45dc-94a6-6e3d5e5bf68b None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:12:11.743509 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6417debf-9da4-4106-a10f-f0bc8318e240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:12:11.744100 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6417debf-9da4-4106-a10f-f0bc8318e240 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:12:11.744379 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6417debf-9da4-4106-a10f-f0bc8318e240 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:12:11.744651 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6417debf-9da4-4106-a10f-f0bc8318e240 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:12:11.745466 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6417debf-9da4-4106-a10f-f0bc8318e240 None admin] RBAC: Adding query filter params (role.id=60af5088793e4379b88821a86552a344, scope.project.id=8e912761be6448d1a82682d8d1054a80, user.id=0580afe7e0d249bca0859ce781e6a66c) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:12:11.745771 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6417debf-9da4-4106-a10f-f0bc8318e240 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:12:11.747184 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6417debf-9da4-4106-a10f-f0bc8318e240 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:12:11.752919 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 573/1146] 10.222.0.58 () {56 vars in 1374 bytes} [Tue May 17 19:12:11 2022] GET /identity/v3/role_assignments?user.id=0580afe7e0d249bca0859ce781e6a66c&scope.project.id=8e912761be6448d1a82682d8d1054a80&role.id=60af5088793e4379b88821a86552a344 => generated 259 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:12:13.897011 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e65a27-b4f6-467f-a712-3307222c2155 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:12:13.897565 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e65a27-b4f6-467f-a712-3307222c2155 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:12:13.897565 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e65a27-b4f6-467f-a712-3307222c2155 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:12:13.898907 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 574/1147] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:12:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:12:13.904110 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277a132c-ac4f-449e-9d5a-cb6dbfb50f99 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:12:13.904486 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277a132c-ac4f-449e-9d5a-cb6dbfb50f99 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:12:13.904657 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277a132c-ac4f-449e-9d5a-cb6dbfb50f99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:12:13.920636 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-277a132c-ac4f-449e-9d5a-cb6dbfb50f99 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:12:13.922285 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-277a132c-ac4f-449e-9d5a-cb6dbfb50f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:12:13.985773 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 574/1148] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:12:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 17 19:12:14.007711 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e65a27-b4f6-467f-a712-3307222c2155 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:12:14.101806 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e65a27-b4f6-467f-a712-3307222c2155 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:12:14.133656 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6bdc3d-8993-4f22-8490-86f24f567e7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:12:14.134421 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6bdc3d-8993-4f22-8490-86f24f567e7a None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:12:14.134696 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6bdc3d-8993-4f22-8490-86f24f567e7a None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:12:14.134965 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6bdc3d-8993-4f22-8490-86f24f567e7a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:12:14.144480 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6bdc3d-8993-4f22-8490-86f24f567e7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 17 19:12:14.153900 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-ea6bdc3d-8993-4f22-8490-86f24f567e7a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 17 19:12:14.153900 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:12:14.153900 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:12:14.153900 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:12:14.153900 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:12:14.153900 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:12:14.153900 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:12:14.153900 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:12:14.153900 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:12:14.153900 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:12:14.153900 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:12:14.153900 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:12:14.153900 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 17 19:12:14.153900 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) May 17 19:12:14.153900 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 17 19:12:14.153900 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 17 19:12:14.154985 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 17 19:12:14.154985 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 17 19:12:14.154985 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 17 19:12:14.154985 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 17 19:12:14.154985 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:12:14.154985 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:12:14.154985 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:12:14.154985 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:12:14.154985 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:12:14.154985 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:12:14.154985 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:12:14.154985 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:12:14.154985 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:12:14.154985 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:12:14.154985 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:12:14.154985 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:12:14.156585 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:12:14.156585 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:12:14.156585 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:12:14.156585 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:12:14.156585 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 17 19:12:14.156585 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 17 19:12:14.156585 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 17 19:12:14.156585 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 17 19:12:14.156585 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 17 19:12:14.156585 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 17 19:12:14.156585 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 17 19:12:14.156585 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:12:14.156585 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 575/1149] 10.222.0.58 () {56 vars in 1096 bytes} [Tue May 17 19:12:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:12:14.159456 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277a132c-ac4f-449e-9d5a-cb6dbfb50f99 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:12:14.250699 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277a132c-ac4f-449e-9d5a-cb6dbfb50f99 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:12:14.281936 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9367d91-b4f6-41b6-978c-872d4fb7130b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:12:14.282577 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9367d91-b4f6-41b6-978c-872d4fb7130b None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:12:14.282850 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9367d91-b4f6-41b6-978c-872d4fb7130b None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:12:14.283117 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9367d91-b4f6-41b6-978c-872d4fb7130b None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:12:14.283870 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9367d91-b4f6-41b6-978c-872d4fb7130b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:12:14.284167 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9367d91-b4f6-41b6-978c-872d4fb7130b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:12:14.285509 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9367d91-b4f6-41b6-978c-872d4fb7130b None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:12:14.292958 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 575/1150] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 17 19:12:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:12:14.299641 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6bdc3d-8993-4f22-8490-86f24f567e7a None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:12:14.401923 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6bdc3d-8993-4f22-8490-86f24f567e7a None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:12:14.433803 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b54d11-3098-4813-977d-a2bf0225ddd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:12:14.434591 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b54d11-3098-4813-977d-a2bf0225ddd7 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:12:14.434862 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b54d11-3098-4813-977d-a2bf0225ddd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:12:14.435127 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b54d11-3098-4813-977d-a2bf0225ddd7 None admin] PATH_INFO: `/v3/users/0580afe7e0d249bca0859ce781e6a66c` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:12:14.436302 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b54d11-3098-4813-977d-a2bf0225ddd7 None admin] RBAC: Authorizing `identity:get_user(user_id=0580afe7e0d249bca0859ce781e6a66c)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:12:14.437746 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b54d11-3098-4813-977d-a2bf0225ddd7 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:12:14.439330 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 576/1151] 10.222.0.58 () {56 vars in 1177 bytes} [Tue May 17 19:12:14 2022] GET /identity/v3/users/0580afe7e0d249bca0859ce781e6a66c => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:12:14.444959 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9367d91-b4f6-41b6-978c-872d4fb7130b None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:12:14.538070 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9367d91-b4f6-41b6-978c-872d4fb7130b None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:12:14.568391 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ee703e-541e-4f94-9248-4c442a81c11f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:12:14.569217 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ee703e-541e-4f94-9248-4c442a81c11f None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:12:14.569489 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ee703e-541e-4f94-9248-4c442a81c11f None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:12:14.569751 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ee703e-541e-4f94-9248-4c442a81c11f None admin] PATH_INFO: `/v3/projects/8e912761be6448d1a82682d8d1054a80` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:12:14.570884 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ee703e-541e-4f94-9248-4c442a81c11f None admin] RBAC: Authorizing `identity:get_project(project_id=8e912761be6448d1a82682d8d1054a80)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:12:14.572309 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ee703e-541e-4f94-9248-4c442a81c11f None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:12:14.574231 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 576/1152] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 17 19:12:14 2022] GET /identity/v3/projects/8e912761be6448d1a82682d8d1054a80 => generated 311 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:12:14.580065 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b54d11-3098-4813-977d-a2bf0225ddd7 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:12:14.672338 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b54d11-3098-4813-977d-a2bf0225ddd7 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:12:14.703593 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74326e2-b84e-4319-80b3-60126b5da2b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:12:14.704443 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74326e2-b84e-4319-80b3-60126b5da2b7 None admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:12:14.704715 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74326e2-b84e-4319-80b3-60126b5da2b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:12:14.704980 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74326e2-b84e-4319-80b3-60126b5da2b7 None admin] PATH_INFO: `/v3/projects/8e912761be6448d1a82682d8d1054a80/users/0580afe7e0d249bca0859ce781e6a66c/roles/60af5088793e4379b88821a86552a344` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:12:14.706876 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74326e2-b84e-4319-80b3-60126b5da2b7 None admin] RBAC: Authorizing `identity:create_grant(project_id=8e912761be6448d1a82682d8d1054a80, user_id=0580afe7e0d249bca0859ce781e6a66c, role_id=60af5088793e4379b88821a86552a344)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:12:14.708382 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74326e2-b84e-4319-80b3-60126b5da2b7 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:12:14.804767 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 577/1153] 10.222.0.58 () {58 vars in 1439 bytes} [Tue May 17 19:12:14 2022] PUT /identity/v3/projects/8e912761be6448d1a82682d8d1054a80/users/0580afe7e0d249bca0859ce781e6a66c/roles/60af5088793e4379b88821a86552a344 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:12:16.951466 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa4cc00-2357-4166-a705-132bca184593 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:12:16.951902 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa4cc00-2357-4166-a705-132bca184593 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:12:16.952051 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa4cc00-2357-4166-a705-132bca184593 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:12:16.953416 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 577/1154] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:12:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:12:16.958695 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d8f34f-0a11-42d1-af66-90996336f7d8 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:12:16.959114 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d8f34f-0a11-42d1-af66-90996336f7d8 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:12:16.959216 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d8f34f-0a11-42d1-af66-90996336f7d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:12:16.975653 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-b7d8f34f-0a11-42d1-af66-90996336f7d8 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:12:16.976816 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b7d8f34f-0a11-42d1-af66-90996336f7d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:12:17.038905 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 578/1155] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:12:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 17 19:12:17.061445 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa4cc00-2357-4166-a705-132bca184593 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:12:17.157104 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa4cc00-2357-4166-a705-132bca184593 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:12:17.188412 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2163eaf-b1bf-482c-96b9-acac0367fa48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:12:17.189171 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2163eaf-b1bf-482c-96b9-acac0367fa48 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:12:17.189449 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2163eaf-b1bf-482c-96b9-acac0367fa48 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:12:17.189715 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2163eaf-b1bf-482c-96b9-acac0367fa48 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:12:17.199644 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2163eaf-b1bf-482c-96b9-acac0367fa48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 17 19:12:17.209347 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-b2163eaf-b1bf-482c-96b9-acac0367fa48 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 17 19:12:17.209347 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:12:17.209347 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:12:17.209347 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:12:17.209347 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:12:17.209347 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:12:17.209347 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:12:17.209347 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:12:17.209347 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:12:17.209347 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:12:17.209347 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:12:17.209347 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:12:17.209347 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 17 19:12:17.209347 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) May 17 19:12:17.209347 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 17 19:12:17.209347 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 17 19:12:17.210519 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 17 19:12:17.210519 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 17 19:12:17.210519 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 17 19:12:17.210519 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 17 19:12:17.210519 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:12:17.210519 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:12:17.210519 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:12:17.210519 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:12:17.210519 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:12:17.210519 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:12:17.210519 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:12:17.210519 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:12:17.210519 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:12:17.210519 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:12:17.210519 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:12:17.210519 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:12:17.211640 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:12:17.211640 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:12:17.211640 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:12:17.211640 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:12:17.211640 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 17 19:12:17.211640 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 17 19:12:17.211640 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 17 19:12:17.211640 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 17 19:12:17.211640 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 17 19:12:17.211640 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 17 19:12:17.211640 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 17 19:12:17.211640 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:12:17.211640 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 578/1156] 10.222.0.58 () {56 vars in 1096 bytes} [Tue May 17 19:12:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:12:17.216257 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d8f34f-0a11-42d1-af66-90996336f7d8 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:12:17.308187 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d8f34f-0a11-42d1-af66-90996336f7d8 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:12:17.341411 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ea0fdd-4688-481e-81f6-69942a7f132d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:12:17.342135 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ea0fdd-4688-481e-81f6-69942a7f132d None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:12:17.342415 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ea0fdd-4688-481e-81f6-69942a7f132d None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:12:17.342727 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ea0fdd-4688-481e-81f6-69942a7f132d None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:12:17.343518 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ea0fdd-4688-481e-81f6-69942a7f132d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:12:17.343816 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ea0fdd-4688-481e-81f6-69942a7f132d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:12:17.345500 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ea0fdd-4688-481e-81f6-69942a7f132d None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:12:17.353548 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 579/1157] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 17 19:12:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:12:17.359947 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2163eaf-b1bf-482c-96b9-acac0367fa48 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:12:17.453013 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2163eaf-b1bf-482c-96b9-acac0367fa48 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:12:17.484531 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d99b1e-f7be-4a01-8793-c4e747fd7dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:12:17.485366 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d99b1e-f7be-4a01-8793-c4e747fd7dda None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:12:17.485639 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d99b1e-f7be-4a01-8793-c4e747fd7dda None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:12:17.485904 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d99b1e-f7be-4a01-8793-c4e747fd7dda None admin] PATH_INFO: `/v3/users/0580afe7e0d249bca0859ce781e6a66c` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:12:17.487059 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d99b1e-f7be-4a01-8793-c4e747fd7dda None admin] RBAC: Authorizing `identity:get_user(user_id=0580afe7e0d249bca0859ce781e6a66c)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:12:17.488493 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d99b1e-f7be-4a01-8793-c4e747fd7dda None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:12:17.490073 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 579/1158] 10.222.0.58 () {56 vars in 1177 bytes} [Tue May 17 19:12:17 2022] GET /identity/v3/users/0580afe7e0d249bca0859ce781e6a66c => generated 286 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:12:17.495729 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ea0fdd-4688-481e-81f6-69942a7f132d None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:12:17.588426 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ea0fdd-4688-481e-81f6-69942a7f132d None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:12:17.619893 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7779f778-c2ee-4dcb-b1e2-d6fa8eec0857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:12:17.620517 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7779f778-c2ee-4dcb-b1e2-d6fa8eec0857 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:12:17.620788 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7779f778-c2ee-4dcb-b1e2-d6fa8eec0857 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:12:17.621054 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7779f778-c2ee-4dcb-b1e2-d6fa8eec0857 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:12:17.683083 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7779f778-c2ee-4dcb-b1e2-d6fa8eec0857 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:12:17.684529 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7779f778-c2ee-4dcb-b1e2-d6fa8eec0857 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:12:17.775455 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 580/1159] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 17 19:12:17 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 17 19:12:17.781861 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d99b1e-f7be-4a01-8793-c4e747fd7dda None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:12:17.874589 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d99b1e-f7be-4a01-8793-c4e747fd7dda None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:12:17.906302 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc96908-b4dc-414f-8e16-58a68c9ba4da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:12:17.907130 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc96908-b4dc-414f-8e16-58a68c9ba4da None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:12:17.907407 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc96908-b4dc-414f-8e16-58a68c9ba4da None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:12:17.907707 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc96908-b4dc-414f-8e16-58a68c9ba4da None admin] PATH_INFO: `/v3/projects/8e912761be6448d1a82682d8d1054a80` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:12:17.908863 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc96908-b4dc-414f-8e16-58a68c9ba4da None admin] RBAC: Authorizing `identity:get_project(project_id=8e912761be6448d1a82682d8d1054a80)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:12:17.910283 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc96908-b4dc-414f-8e16-58a68c9ba4da None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:12:17.911992 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 580/1160] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 17 19:12:17 2022] GET /identity/v3/projects/8e912761be6448d1a82682d8d1054a80 => generated 311 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:12:17.917974 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7779f778-c2ee-4dcb-b1e2-d6fa8eec0857 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:12:18.011632 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7779f778-c2ee-4dcb-b1e2-d6fa8eec0857 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:12:18.043849 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275274b5-3e8f-4e16-a38b-a7b38154ef1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:12:18.044427 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275274b5-3e8f-4e16-a38b-a7b38154ef1d None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:12:18.044704 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275274b5-3e8f-4e16-a38b-a7b38154ef1d None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:12:18.044979 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275274b5-3e8f-4e16-a38b-a7b38154ef1d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:12:18.045786 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275274b5-3e8f-4e16-a38b-a7b38154ef1d None admin] RBAC: Adding query filter params (role.id=60af5088793e4379b88821a86552a344, scope.project.id=8e912761be6448d1a82682d8d1054a80, user.id=0580afe7e0d249bca0859ce781e6a66c) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:12:18.046113 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275274b5-3e8f-4e16-a38b-a7b38154ef1d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:12:18.047545 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275274b5-3e8f-4e16-a38b-a7b38154ef1d None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:12:18.053111 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 581/1161] 10.222.0.58 () {56 vars in 1374 bytes} [Tue May 17 19:12:17 2022] GET /identity/v3/role_assignments?user.id=0580afe7e0d249bca0859ce781e6a66c&scope.project.id=8e912761be6448d1a82682d8d1054a80&role.id=60af5088793e4379b88821a86552a344 => generated 608 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:12:20.242374 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595d100d-6c94-484e-b45b-4a59d617364c None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:12:20.242880 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595d100d-6c94-484e-b45b-4a59d617364c None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:12:20.242880 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595d100d-6c94-484e-b45b-4a59d617364c None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:12:20.244236 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 581/1162] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:12:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:12:20.249736 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77413b47-09ab-4000-9999-9ee3abed229d None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:12:20.250174 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77413b47-09ab-4000-9999-9ee3abed229d None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:12:20.250273 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77413b47-09ab-4000-9999-9ee3abed229d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:12:20.267522 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-77413b47-09ab-4000-9999-9ee3abed229d None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:12:20.268702 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-77413b47-09ab-4000-9999-9ee3abed229d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:12:20.331073 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 582/1163] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:12:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 17 19:12:20.353369 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595d100d-6c94-484e-b45b-4a59d617364c None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:12:20.448150 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595d100d-6c94-484e-b45b-4a59d617364c None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:12:20.479849 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe88566-43b6-43b1-b1e5-6aaac781689f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:12:20.480465 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe88566-43b6-43b1-b1e5-6aaac781689f None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:12:20.480743 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe88566-43b6-43b1-b1e5-6aaac781689f None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:12:20.481017 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe88566-43b6-43b1-b1e5-6aaac781689f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:12:20.542730 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe88566-43b6-43b1-b1e5-6aaac781689f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:12:20.544128 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe88566-43b6-43b1-b1e5-6aaac781689f None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:12:20.634988 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 582/1164] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 17 19:12:20 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 17 19:12:20.641331 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77413b47-09ab-4000-9999-9ee3abed229d None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:12:20.732912 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77413b47-09ab-4000-9999-9ee3abed229d None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:12:20.763648 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac59190-f4e3-4a09-b876-93498685eaf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:12:20.764943 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac59190-f4e3-4a09-b876-93498685eaf2 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:12:20.765219 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac59190-f4e3-4a09-b876-93498685eaf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:12:20.765485 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac59190-f4e3-4a09-b876-93498685eaf2 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:12:20.771827 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac59190-f4e3-4a09-b876-93498685eaf2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:12:20.773260 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac59190-f4e3-4a09-b876-93498685eaf2 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:12:20.788199 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-dac59190-f4e3-4a09-b876-93498685eaf2 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. May 17 19:12:20.788199 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:12:20.788199 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 17 19:12:20.788199 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 17 19:12:20.788199 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 17 19:12:20.788199 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 17 19:12:20.788199 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 17 19:12:20.788199 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 17 19:12:20.788199 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. May 17 19:12:20.788199 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:12:20.788199 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 17 19:12:20.788199 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:12:20.788199 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:12:20.788199 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:12:20.788199 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:12:20.788199 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:12:20.788199 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:12:20.789473 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:12:20.789473 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:12:20.789473 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:12:20.789473 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:12:20.789473 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:12:20.789473 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:12:20.789473 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 17 19:12:20.789473 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 17 19:12:20.789473 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 17 19:12:20.789473 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 17 19:12:20.789473 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:12:20.789473 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:12:20.789473 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:12:20.789473 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:12:20.789473 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:12:20.789473 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:12:20.790682 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:12:20.790682 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:12:20.790682 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:12:20.790682 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:12:20.790682 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:12:20.790682 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:12:20.790682 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:12:20.790682 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:12:20.790682 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:12:20.790682 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:12:20.790682 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 17 19:12:20.790682 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 17 19:12:20.790682 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. May 17 19:12:20.790682 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:12:20.790682 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 583/1165] 10.222.0.58 () {56 vars in 1117 bytes} [Tue May 17 19:12:20 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:12:20.795173 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe88566-43b6-43b1-b1e5-6aaac781689f None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:12:20.887282 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe88566-43b6-43b1-b1e5-6aaac781689f None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:12:20.919116 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd5b1fa-dfb5-4711-9737-1bb9f49d80e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:12:20.919790 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd5b1fa-dfb5-4711-9737-1bb9f49d80e9 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:12:20.920068 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd5b1fa-dfb5-4711-9737-1bb9f49d80e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:12:20.920336 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd5b1fa-dfb5-4711-9737-1bb9f49d80e9 None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:12:20.921043 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd5b1fa-dfb5-4711-9737-1bb9f49d80e9 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:12:20.921341 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd5b1fa-dfb5-4711-9737-1bb9f49d80e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:12:20.922685 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd5b1fa-dfb5-4711-9737-1bb9f49d80e9 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:12:20.929427 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 583/1166] 10.222.0.58 () {56 vars in 1115 bytes} [Tue May 17 19:12:20 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:12:23.079338 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88429631-d0d0-49f5-9630-c05605e9466d None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:12:23.079895 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88429631-d0d0-49f5-9630-c05605e9466d None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:12:23.079895 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88429631-d0d0-49f5-9630-c05605e9466d None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:12:23.081263 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 584/1167] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:12:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:12:23.086520 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b46de9-dcd0-4574-9b76-7bf7d16a2204 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:12:23.086905 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b46de9-dcd0-4574-9b76-7bf7d16a2204 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:12:23.087075 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b46de9-dcd0-4574-9b76-7bf7d16a2204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:12:23.103902 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-28b46de9-dcd0-4574-9b76-7bf7d16a2204 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:12:23.105070 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-28b46de9-dcd0-4574-9b76-7bf7d16a2204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:12:23.166757 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 584/1168] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:12:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 17 19:12:23.189743 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88429631-d0d0-49f5-9630-c05605e9466d None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:12:23.285987 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88429631-d0d0-49f5-9630-c05605e9466d None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:12:23.316790 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9d8e17-ed97-4cbf-8265-5db9be70960e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:12:23.317424 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9d8e17-ed97-4cbf-8265-5db9be70960e None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:12:23.317697 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9d8e17-ed97-4cbf-8265-5db9be70960e None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:12:23.317962 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9d8e17-ed97-4cbf-8265-5db9be70960e None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:12:23.318688 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9d8e17-ed97-4cbf-8265-5db9be70960e None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:12:23.320058 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9d8e17-ed97-4cbf-8265-5db9be70960e None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:12:23.440788 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 585/1169] 10.222.0.58 () {60 vars in 1138 bytes} [Tue May 17 19:12:23 2022] POST /identity/v3/domains => generated 267 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:12:25.622311 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f16ff2-1b55-4418-8c8c-7abf2f2b2f04 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:12:25.622860 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f16ff2-1b55-4418-8c8c-7abf2f2b2f04 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:12:25.622860 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f16ff2-1b55-4418-8c8c-7abf2f2b2f04 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:12:25.624750 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 585/1170] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:12:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:12:25.629855 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8903d409-435f-4ce2-8012-d413b873e7e4 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:12:25.630248 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8903d409-435f-4ce2-8012-d413b873e7e4 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:12:25.630402 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8903d409-435f-4ce2-8012-d413b873e7e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:12:25.647141 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-8903d409-435f-4ce2-8012-d413b873e7e4 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:12:25.648343 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-8903d409-435f-4ce2-8012-d413b873e7e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:12:25.710961 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 586/1171] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:12:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 17 19:12:25.734247 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f16ff2-1b55-4418-8c8c-7abf2f2b2f04 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:12:25.828504 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f16ff2-1b55-4418-8c8c-7abf2f2b2f04 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:12:25.860323 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bcf6b5-ddd4-4a68-91b0-b2e95f92b82d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:12:25.861159 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bcf6b5-ddd4-4a68-91b0-b2e95f92b82d None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:12:25.861432 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bcf6b5-ddd4-4a68-91b0-b2e95f92b82d None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:12:25.861700 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bcf6b5-ddd4-4a68-91b0-b2e95f92b82d None admin] PATH_INFO: `/v3/domains/d1588bb70a88484999b21cdfe338a253` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:12:25.871664 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bcf6b5-ddd4-4a68-91b0-b2e95f92b82d None admin] RBAC: Authorizing `identity:get_domain(domain_id=d1588bb70a88484999b21cdfe338a253)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:12:25.873128 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bcf6b5-ddd4-4a68-91b0-b2e95f92b82d None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:12:25.874722 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 586/1172] 10.222.0.58 () {56 vars in 1183 bytes} [Tue May 17 19:12:25 2022] GET /identity/v3/domains/d1588bb70a88484999b21cdfe338a253 => generated 267 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:12:25.880415 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8903d409-435f-4ce2-8012-d413b873e7e4 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:12:25.973397 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8903d409-435f-4ce2-8012-d413b873e7e4 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:12:26.004742 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538993e1-e168-42fa-a31f-36aecb61e45c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:12:26.005385 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538993e1-e168-42fa-a31f-36aecb61e45c None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:12:26.005665 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538993e1-e168-42fa-a31f-36aecb61e45c None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:12:26.005937 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538993e1-e168-42fa-a31f-36aecb61e45c None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:12:26.006638 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538993e1-e168-42fa-a31f-36aecb61e45c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:12:26.008103 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538993e1-e168-42fa-a31f-36aecb61e45c None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:12:26.097308 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 587/1173] 10.222.0.58 () {60 vars in 1141 bytes} [Tue May 17 19:12:25 2022] POST /identity/v3/projects => generated 361 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:12:28.289001 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a76b792-c16e-4e26-82d9-c6418fffc439 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:12:28.289502 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a76b792-c16e-4e26-82d9-c6418fffc439 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:12:28.289502 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a76b792-c16e-4e26-82d9-c6418fffc439 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:12:28.290820 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 587/1174] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:12:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:12:28.296149 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d175bd-e6d4-4d15-be40-704701cb04fe None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:12:28.296568 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d175bd-e6d4-4d15-be40-704701cb04fe None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:12:28.296702 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d175bd-e6d4-4d15-be40-704701cb04fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:12:28.313457 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-e3d175bd-e6d4-4d15-be40-704701cb04fe None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:12:28.314636 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-e3d175bd-e6d4-4d15-be40-704701cb04fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:12:28.377649 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 588/1175] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:12:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 17 19:12:28.402030 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a76b792-c16e-4e26-82d9-c6418fffc439 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:12:28.497503 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a76b792-c16e-4e26-82d9-c6418fffc439 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:12:28.529296 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b20a3e-75b6-49e2-9201-d98305116da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:12:28.530159 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b20a3e-75b6-49e2-9201-d98305116da1 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:12:28.530436 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b20a3e-75b6-49e2-9201-d98305116da1 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:12:28.530704 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b20a3e-75b6-49e2-9201-d98305116da1 None admin] PATH_INFO: `/v3/domains/d1588bb70a88484999b21cdfe338a253` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:12:28.531891 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b20a3e-75b6-49e2-9201-d98305116da1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=d1588bb70a88484999b21cdfe338a253)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:12:28.533307 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b20a3e-75b6-49e2-9201-d98305116da1 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:12:28.534895 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 588/1176] 10.222.0.58 () {56 vars in 1183 bytes} [Tue May 17 19:12:28 2022] GET /identity/v3/domains/d1588bb70a88484999b21cdfe338a253 => generated 267 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:12:28.540664 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d175bd-e6d4-4d15-be40-704701cb04fe None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:12:28.632655 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d175bd-e6d4-4d15-be40-704701cb04fe None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:12:28.664251 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b94348-07d2-4ccd-bcc1-c159d2794ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:12:28.664895 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b94348-07d2-4ccd-bcc1-c159d2794ec7 None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:12:28.665170 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b94348-07d2-4ccd-bcc1-c159d2794ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:12:28.665437 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b94348-07d2-4ccd-bcc1-c159d2794ec7 None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:12:28.666124 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b94348-07d2-4ccd-bcc1-c159d2794ec7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:12:28.667578 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b94348-07d2-4ccd-bcc1-c159d2794ec7 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:12:28.761454 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 589/1177] 10.222.0.58 () {60 vars in 1132 bytes} [Tue May 17 19:12:28 2022] POST /identity/v3/users => generated 311 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:12:30.963365 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707d2995-00ce-4bb2-8c65-a51000c155d7 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:12:30.963791 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707d2995-00ce-4bb2-8c65-a51000c155d7 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:12:30.963942 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707d2995-00ce-4bb2-8c65-a51000c155d7 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:12:30.965290 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 589/1178] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:12:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:12:30.970724 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013c6b57-5989-47e7-b6f4-6550a8d8fa7d None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:12:30.971147 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013c6b57-5989-47e7-b6f4-6550a8d8fa7d None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:12:30.971249 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013c6b57-5989-47e7-b6f4-6550a8d8fa7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:12:30.987847 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-013c6b57-5989-47e7-b6f4-6550a8d8fa7d None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:12:30.989007 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-013c6b57-5989-47e7-b6f4-6550a8d8fa7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:12:31.050537 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 590/1179] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:12:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 17 19:12:31.073524 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707d2995-00ce-4bb2-8c65-a51000c155d7 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:12:31.168611 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707d2995-00ce-4bb2-8c65-a51000c155d7 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:12:31.199538 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e21e3f-98d4-4c23-8ff2-f07c66f69b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:12:31.200243 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e21e3f-98d4-4c23-8ff2-f07c66f69b72 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:12:31.200521 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e21e3f-98d4-4c23-8ff2-f07c66f69b72 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:12:31.200789 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e21e3f-98d4-4c23-8ff2-f07c66f69b72 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:12:31.211122 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e21e3f-98d4-4c23-8ff2-f07c66f69b72 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 17 19:12:31.218629 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-39e21e3f-98d4-4c23-8ff2-f07c66f69b72 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 17 19:12:31.218629 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:12:31.218629 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:12:31.218629 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:12:31.218629 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:12:31.218629 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:12:31.218629 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:12:31.218629 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:12:31.218629 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:12:31.218629 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:12:31.218629 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:12:31.218629 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:12:31.218629 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 17 19:12:31.218629 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) May 17 19:12:31.218629 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 17 19:12:31.218629 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 17 19:12:31.219759 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 17 19:12:31.219759 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 17 19:12:31.219759 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 17 19:12:31.219759 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 17 19:12:31.219759 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:12:31.219759 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:12:31.219759 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:12:31.219759 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:12:31.219759 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:12:31.219759 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:12:31.219759 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:12:31.219759 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:12:31.219759 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:12:31.219759 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:12:31.219759 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:12:31.219759 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:12:31.221068 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:12:31.221068 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:12:31.221068 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:12:31.221068 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:12:31.221068 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 17 19:12:31.221068 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 17 19:12:31.221068 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 17 19:12:31.221068 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 17 19:12:31.221068 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 17 19:12:31.221068 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 17 19:12:31.221068 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 17 19:12:31.221068 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:12:31.221068 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 590/1180] 10.222.0.58 () {56 vars in 1096 bytes} [Tue May 17 19:12:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:12:31.225759 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013c6b57-5989-47e7-b6f4-6550a8d8fa7d None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:12:31.317642 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013c6b57-5989-47e7-b6f4-6550a8d8fa7d None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:12:31.349218 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20782cf3-1232-4639-bda1-9595160c9426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:12:31.349861 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20782cf3-1232-4639-bda1-9595160c9426 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:12:31.350138 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20782cf3-1232-4639-bda1-9595160c9426 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:12:31.350408 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20782cf3-1232-4639-bda1-9595160c9426 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:12:31.351127 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20782cf3-1232-4639-bda1-9595160c9426 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:12:31.351424 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20782cf3-1232-4639-bda1-9595160c9426 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:12:31.352798 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20782cf3-1232-4639-bda1-9595160c9426 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:12:31.360396 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 591/1181] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 17 19:12:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:12:31.365943 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e21e3f-98d4-4c23-8ff2-f07c66f69b72 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:12:31.457864 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e21e3f-98d4-4c23-8ff2-f07c66f69b72 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:12:31.489612 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863166ed-84a2-4f0f-b803-59273fee88da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:12:31.490445 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863166ed-84a2-4f0f-b803-59273fee88da None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:12:31.490719 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863166ed-84a2-4f0f-b803-59273fee88da None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:12:31.490986 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863166ed-84a2-4f0f-b803-59273fee88da None admin] PATH_INFO: `/v3/users/4a8260c60c884d5080cd3d81e993486b` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:12:31.503720 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863166ed-84a2-4f0f-b803-59273fee88da None admin] RBAC: Authorizing `identity:get_user(user_id=4a8260c60c884d5080cd3d81e993486b)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:12:31.505138 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863166ed-84a2-4f0f-b803-59273fee88da None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:12:31.506770 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 591/1182] 10.222.0.58 () {56 vars in 1177 bytes} [Tue May 17 19:12:31 2022] GET /identity/v3/users/4a8260c60c884d5080cd3d81e993486b => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:12:31.512439 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20782cf3-1232-4639-bda1-9595160c9426 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:12:31.605560 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20782cf3-1232-4639-bda1-9595160c9426 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:12:31.636723 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20c48fe-95dd-4f90-aeb3-324fce5457b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:12:31.637346 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20c48fe-95dd-4f90-aeb3-324fce5457b2 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:12:31.637652 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20c48fe-95dd-4f90-aeb3-324fce5457b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:12:31.637923 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20c48fe-95dd-4f90-aeb3-324fce5457b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:12:31.700133 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20c48fe-95dd-4f90-aeb3-324fce5457b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:12:31.701659 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20c48fe-95dd-4f90-aeb3-324fce5457b2 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:12:31.843821 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 592/1183] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 17 19:12:31 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 17 19:12:31.849961 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863166ed-84a2-4f0f-b803-59273fee88da None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:12:31.941380 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863166ed-84a2-4f0f-b803-59273fee88da None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:12:31.973484 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d30ae7-c053-43d5-b4d4-391386960010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:12:31.974320 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d30ae7-c053-43d5-b4d4-391386960010 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:12:31.974592 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d30ae7-c053-43d5-b4d4-391386960010 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:12:31.974856 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d30ae7-c053-43d5-b4d4-391386960010 None admin] PATH_INFO: `/v3/projects/51dd7f217f1c418ba22e91f310b29c74` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:12:31.976021 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d30ae7-c053-43d5-b4d4-391386960010 None admin] RBAC: Authorizing `identity:get_project(project_id=51dd7f217f1c418ba22e91f310b29c74)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:12:31.977454 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d30ae7-c053-43d5-b4d4-391386960010 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:12:31.979101 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 592/1184] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 17 19:12:31 2022] GET /identity/v3/projects/51dd7f217f1c418ba22e91f310b29c74 => generated 361 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:12:31.985030 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20c48fe-95dd-4f90-aeb3-324fce5457b2 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:12:32.077529 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20c48fe-95dd-4f90-aeb3-324fce5457b2 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:12:32.109635 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ce06bd-7df1-4b2b-985b-5040a45d5a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:12:32.110204 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ce06bd-7df1-4b2b-985b-5040a45d5a0e None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:12:32.110478 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ce06bd-7df1-4b2b-985b-5040a45d5a0e None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:12:32.110744 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ce06bd-7df1-4b2b-985b-5040a45d5a0e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:12:32.111575 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ce06bd-7df1-4b2b-985b-5040a45d5a0e None admin] RBAC: Adding query filter params (role.id=60af5088793e4379b88821a86552a344, scope.project.id=51dd7f217f1c418ba22e91f310b29c74, user.id=4a8260c60c884d5080cd3d81e993486b) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:12:32.111880 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ce06bd-7df1-4b2b-985b-5040a45d5a0e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:12:32.113293 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ce06bd-7df1-4b2b-985b-5040a45d5a0e None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:12:32.118956 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 593/1185] 10.222.0.58 () {56 vars in 1374 bytes} [Tue May 17 19:12:31 2022] GET /identity/v3/role_assignments?user.id=4a8260c60c884d5080cd3d81e993486b&scope.project.id=51dd7f217f1c418ba22e91f310b29c74&role.id=60af5088793e4379b88821a86552a344 => generated 259 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:12:34.261638 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505b7433-6e9f-4901-a4f0-01775a6ebcdb None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:12:34.262142 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505b7433-6e9f-4901-a4f0-01775a6ebcdb None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:12:34.262142 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505b7433-6e9f-4901-a4f0-01775a6ebcdb None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:12:34.263441 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 593/1186] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:12:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:12:34.269001 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b9a924-8403-4c32-86b9-71fe71aacc0b None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:12:34.269424 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b9a924-8403-4c32-86b9-71fe71aacc0b None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:12:34.269552 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b9a924-8403-4c32-86b9-71fe71aacc0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:12:34.286344 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-e6b9a924-8403-4c32-86b9-71fe71aacc0b None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:12:34.287549 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-e6b9a924-8403-4c32-86b9-71fe71aacc0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:12:34.349815 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 594/1187] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:12:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 17 19:12:34.371574 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505b7433-6e9f-4901-a4f0-01775a6ebcdb None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:12:34.465343 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505b7433-6e9f-4901-a4f0-01775a6ebcdb None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:12:34.496738 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07df8dd7-f8bd-497e-b619-1b59e6b7cdd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:12:34.497502 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07df8dd7-f8bd-497e-b619-1b59e6b7cdd6 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:12:34.497781 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07df8dd7-f8bd-497e-b619-1b59e6b7cdd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:12:34.498545 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07df8dd7-f8bd-497e-b619-1b59e6b7cdd6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:12:34.508076 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07df8dd7-f8bd-497e-b619-1b59e6b7cdd6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 17 19:12:34.516189 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-07df8dd7-f8bd-497e-b619-1b59e6b7cdd6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 17 19:12:34.516189 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:12:34.516189 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:12:34.516189 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:12:34.516189 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:12:34.516189 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:12:34.516189 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:12:34.516189 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:12:34.516189 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:12:34.516189 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:12:34.516189 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:12:34.516189 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:12:34.516189 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 17 19:12:34.516189 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) May 17 19:12:34.516189 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 17 19:12:34.516189 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 17 19:12:34.517344 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 17 19:12:34.517344 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 17 19:12:34.517344 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 17 19:12:34.517344 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 17 19:12:34.517344 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:12:34.517344 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:12:34.517344 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:12:34.517344 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:12:34.517344 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:12:34.517344 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:12:34.517344 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:12:34.517344 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:12:34.517344 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:12:34.517344 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:12:34.517344 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:12:34.517344 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:12:34.518179 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:12:34.518179 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:12:34.518179 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:12:34.518179 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:12:34.518179 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 17 19:12:34.518179 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 17 19:12:34.518179 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 17 19:12:34.518179 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 17 19:12:34.518179 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 17 19:12:34.518179 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 17 19:12:34.518179 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 17 19:12:34.518179 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:12:34.518179 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 594/1188] 10.222.0.58 () {56 vars in 1096 bytes} [Tue May 17 19:12:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:12:34.523235 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b9a924-8403-4c32-86b9-71fe71aacc0b None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:12:34.616053 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b9a924-8403-4c32-86b9-71fe71aacc0b None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:12:34.648037 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d415b5-e864-44bd-9b3c-9f2fce4283ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:12:34.648687 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d415b5-e864-44bd-9b3c-9f2fce4283ef None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:12:34.648965 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d415b5-e864-44bd-9b3c-9f2fce4283ef None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:12:34.649238 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d415b5-e864-44bd-9b3c-9f2fce4283ef None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:12:34.649966 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d415b5-e864-44bd-9b3c-9f2fce4283ef None admin] RBAC: Adding query filter params (name=admin) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:12:34.650268 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d415b5-e864-44bd-9b3c-9f2fce4283ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:12:34.651696 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d415b5-e864-44bd-9b3c-9f2fce4283ef None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:12:34.660743 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 595/1189] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 17 19:12:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:12:34.666613 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07df8dd7-f8bd-497e-b619-1b59e6b7cdd6 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:12:34.761143 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07df8dd7-f8bd-497e-b619-1b59e6b7cdd6 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:12:34.796594 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db9800b-ba90-4b1e-b4b6-b8447a352b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:12:34.797371 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db9800b-ba90-4b1e-b4b6-b8447a352b70 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:12:34.797649 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db9800b-ba90-4b1e-b4b6-b8447a352b70 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:12:34.797920 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db9800b-ba90-4b1e-b4b6-b8447a352b70 None admin] PATH_INFO: `/v3/users/4a8260c60c884d5080cd3d81e993486b` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:12:34.799087 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db9800b-ba90-4b1e-b4b6-b8447a352b70 None admin] RBAC: Authorizing `identity:get_user(user_id=4a8260c60c884d5080cd3d81e993486b)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:12:34.800532 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db9800b-ba90-4b1e-b4b6-b8447a352b70 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:12:34.802139 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 595/1190] 10.222.0.58 () {56 vars in 1177 bytes} [Tue May 17 19:12:34 2022] GET /identity/v3/users/4a8260c60c884d5080cd3d81e993486b => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:12:34.807787 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d415b5-e864-44bd-9b3c-9f2fce4283ef None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:12:34.903205 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d415b5-e864-44bd-9b3c-9f2fce4283ef None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:12:34.934486 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a34254e-50e8-42b6-a2e0-17317cfa5f2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:12:34.935324 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a34254e-50e8-42b6-a2e0-17317cfa5f2c None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:12:34.935646 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a34254e-50e8-42b6-a2e0-17317cfa5f2c None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:12:34.935911 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a34254e-50e8-42b6-a2e0-17317cfa5f2c None admin] PATH_INFO: `/v3/projects/51dd7f217f1c418ba22e91f310b29c74` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:12:34.937054 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a34254e-50e8-42b6-a2e0-17317cfa5f2c None admin] RBAC: Authorizing `identity:get_project(project_id=51dd7f217f1c418ba22e91f310b29c74)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:12:34.938463 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a34254e-50e8-42b6-a2e0-17317cfa5f2c None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:12:34.940143 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 596/1191] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 17 19:12:34 2022] GET /identity/v3/projects/51dd7f217f1c418ba22e91f310b29c74 => generated 361 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:12:34.945982 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db9800b-ba90-4b1e-b4b6-b8447a352b70 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:12:35.038254 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db9800b-ba90-4b1e-b4b6-b8447a352b70 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:12:35.068915 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1fbc5b-4d3e-4e0c-8747-45d9108419ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:12:35.069711 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1fbc5b-4d3e-4e0c-8747-45d9108419ba None admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:12:35.069986 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1fbc5b-4d3e-4e0c-8747-45d9108419ba None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:12:35.070254 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1fbc5b-4d3e-4e0c-8747-45d9108419ba None admin] PATH_INFO: `/v3/projects/51dd7f217f1c418ba22e91f310b29c74/users/4a8260c60c884d5080cd3d81e993486b/roles/60af5088793e4379b88821a86552a344` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:12:35.072154 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1fbc5b-4d3e-4e0c-8747-45d9108419ba None admin] RBAC: Authorizing `identity:create_grant(project_id=51dd7f217f1c418ba22e91f310b29c74, user_id=4a8260c60c884d5080cd3d81e993486b, role_id=60af5088793e4379b88821a86552a344)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:12:35.073560 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1fbc5b-4d3e-4e0c-8747-45d9108419ba None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:12:35.161251 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 596/1192] 10.222.0.58 () {58 vars in 1439 bytes} [Tue May 17 19:12:34 2022] PUT /identity/v3/projects/51dd7f217f1c418ba22e91f310b29c74/users/4a8260c60c884d5080cd3d81e993486b/roles/60af5088793e4379b88821a86552a344 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:12:37.304897 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0471cfb-efe6-42f6-8fb2-e455eede4e30 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:12:37.305397 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0471cfb-efe6-42f6-8fb2-e455eede4e30 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:12:37.305397 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0471cfb-efe6-42f6-8fb2-e455eede4e30 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:12:37.306706 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 597/1193] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:12:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:12:37.312501 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c76775-46f4-4be3-bb3a-f64c83cd5009 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:12:37.312906 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c76775-46f4-4be3-bb3a-f64c83cd5009 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:12:37.313042 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c76775-46f4-4be3-bb3a-f64c83cd5009 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:12:37.329045 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-c3c76775-46f4-4be3-bb3a-f64c83cd5009 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:12:37.330202 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-c3c76775-46f4-4be3-bb3a-f64c83cd5009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:12:37.391468 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 597/1194] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:12:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 17 19:12:37.414255 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0471cfb-efe6-42f6-8fb2-e455eede4e30 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:12:37.509164 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0471cfb-efe6-42f6-8fb2-e455eede4e30 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:12:37.540053 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878e3f78-5ac8-4be0-a72a-795b828f1668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:12:37.540813 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878e3f78-5ac8-4be0-a72a-795b828f1668 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:12:37.541088 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878e3f78-5ac8-4be0-a72a-795b828f1668 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:12:37.541352 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878e3f78-5ac8-4be0-a72a-795b828f1668 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:12:37.551985 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878e3f78-5ac8-4be0-a72a-795b828f1668 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 17 19:12:37.559407 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-878e3f78-5ac8-4be0-a72a-795b828f1668 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 17 19:12:37.559407 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:12:37.559407 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:12:37.559407 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:12:37.559407 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:12:37.559407 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:12:37.559407 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:12:37.559407 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:12:37.559407 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:12:37.559407 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:12:37.559407 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:12:37.559407 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:12:37.559407 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 17 19:12:37.559407 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) May 17 19:12:37.559407 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 17 19:12:37.559407 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 17 19:12:37.560536 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 17 19:12:37.560536 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 17 19:12:37.560536 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 17 19:12:37.560536 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 17 19:12:37.560536 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:12:37.560536 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:12:37.560536 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:12:37.560536 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:12:37.560536 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:12:37.560536 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:12:37.560536 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:12:37.560536 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:12:37.560536 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:12:37.560536 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:12:37.560536 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:12:37.560536 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:12:37.561574 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:12:37.561574 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:12:37.561574 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:12:37.561574 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:12:37.561574 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 17 19:12:37.561574 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 17 19:12:37.561574 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 17 19:12:37.561574 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 17 19:12:37.561574 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 17 19:12:37.561574 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 17 19:12:37.561574 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 17 19:12:37.561574 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:12:37.561574 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 598/1195] 10.222.0.58 () {56 vars in 1096 bytes} [Tue May 17 19:12:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:12:37.566397 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c76775-46f4-4be3-bb3a-f64c83cd5009 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:12:37.657931 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c76775-46f4-4be3-bb3a-f64c83cd5009 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:12:37.690017 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc1c201-1e1f-4994-b5be-eb888709f98d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:12:37.690694 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc1c201-1e1f-4994-b5be-eb888709f98d None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:12:37.690971 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc1c201-1e1f-4994-b5be-eb888709f98d None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:12:37.691243 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc1c201-1e1f-4994-b5be-eb888709f98d None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:12:37.692006 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc1c201-1e1f-4994-b5be-eb888709f98d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:12:37.692308 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc1c201-1e1f-4994-b5be-eb888709f98d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:12:37.693669 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc1c201-1e1f-4994-b5be-eb888709f98d None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:12:37.701885 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 598/1196] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 17 19:12:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:12:37.707729 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878e3f78-5ac8-4be0-a72a-795b828f1668 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:12:37.801307 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878e3f78-5ac8-4be0-a72a-795b828f1668 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:12:37.835410 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfe9da4-6963-4957-a5bd-508a1e024b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:12:37.836228 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfe9da4-6963-4957-a5bd-508a1e024b00 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:12:37.836504 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfe9da4-6963-4957-a5bd-508a1e024b00 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:12:37.836775 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfe9da4-6963-4957-a5bd-508a1e024b00 None admin] PATH_INFO: `/v3/users/4a8260c60c884d5080cd3d81e993486b` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:12:37.837941 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfe9da4-6963-4957-a5bd-508a1e024b00 None admin] RBAC: Authorizing `identity:get_user(user_id=4a8260c60c884d5080cd3d81e993486b)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:12:37.839364 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfe9da4-6963-4957-a5bd-508a1e024b00 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:12:37.841016 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 599/1197] 10.222.0.58 () {56 vars in 1177 bytes} [Tue May 17 19:12:37 2022] GET /identity/v3/users/4a8260c60c884d5080cd3d81e993486b => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:12:37.847972 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc1c201-1e1f-4994-b5be-eb888709f98d None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:12:37.941925 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc1c201-1e1f-4994-b5be-eb888709f98d None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:12:37.974418 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f4826a-59a6-4d43-9670-231429536f67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:12:37.975048 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f4826a-59a6-4d43-9670-231429536f67 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:12:37.975327 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f4826a-59a6-4d43-9670-231429536f67 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:12:37.975641 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f4826a-59a6-4d43-9670-231429536f67 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:12:38.037561 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f4826a-59a6-4d43-9670-231429536f67 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:12:38.038988 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f4826a-59a6-4d43-9670-231429536f67 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:12:38.129412 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 599/1198] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 17 19:12:37 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 17 19:12:38.135066 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfe9da4-6963-4957-a5bd-508a1e024b00 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:12:38.228527 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfe9da4-6963-4957-a5bd-508a1e024b00 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:12:38.259952 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b799eafa-ec44-4e6e-b746-86d7f778fb88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:12:38.260779 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b799eafa-ec44-4e6e-b746-86d7f778fb88 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:12:38.261053 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b799eafa-ec44-4e6e-b746-86d7f778fb88 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:12:38.261318 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b799eafa-ec44-4e6e-b746-86d7f778fb88 None admin] PATH_INFO: `/v3/projects/51dd7f217f1c418ba22e91f310b29c74` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:12:38.262463 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b799eafa-ec44-4e6e-b746-86d7f778fb88 None admin] RBAC: Authorizing `identity:get_project(project_id=51dd7f217f1c418ba22e91f310b29c74)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:12:38.263955 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b799eafa-ec44-4e6e-b746-86d7f778fb88 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:12:38.265595 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 600/1199] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 17 19:12:38 2022] GET /identity/v3/projects/51dd7f217f1c418ba22e91f310b29c74 => generated 361 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:12:38.271314 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f4826a-59a6-4d43-9670-231429536f67 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:12:38.362781 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f4826a-59a6-4d43-9670-231429536f67 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:12:38.395180 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a30d7e1-fa6a-4206-a79a-12822a40037a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:12:38.395792 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a30d7e1-fa6a-4206-a79a-12822a40037a None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:12:38.396071 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a30d7e1-fa6a-4206-a79a-12822a40037a None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:12:38.396343 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a30d7e1-fa6a-4206-a79a-12822a40037a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:12:38.397162 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a30d7e1-fa6a-4206-a79a-12822a40037a None admin] RBAC: Adding query filter params (role.id=60af5088793e4379b88821a86552a344, scope.project.id=51dd7f217f1c418ba22e91f310b29c74, user.id=4a8260c60c884d5080cd3d81e993486b) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:12:38.397469 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a30d7e1-fa6a-4206-a79a-12822a40037a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:12:38.398881 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a30d7e1-fa6a-4206-a79a-12822a40037a None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:12:38.404490 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 600/1200] 10.222.0.58 () {56 vars in 1374 bytes} [Tue May 17 19:12:38 2022] GET /identity/v3/role_assignments?user.id=4a8260c60c884d5080cd3d81e993486b&scope.project.id=51dd7f217f1c418ba22e91f310b29c74&role.id=60af5088793e4379b88821a86552a344 => generated 608 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:13:54.255948 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0cd3cb-b5ee-4110-9402-a0a75e56d0ef None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:13:54.256568 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0cd3cb-b5ee-4110-9402-a0a75e56d0ef None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:13:54.256568 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0cd3cb-b5ee-4110-9402-a0a75e56d0ef None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:13:54.257773 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 601/1201] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:13:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:13:54.263654 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2441a68-7d23-44a7-be72-81c49c6ced25 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:13:54.264090 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2441a68-7d23-44a7-be72-81c49c6ced25 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:13:54.264250 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2441a68-7d23-44a7-be72-81c49c6ced25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:13:54.281315 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-d2441a68-7d23-44a7-be72-81c49c6ced25 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:13:54.287317 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-d2441a68-7d23-44a7-be72-81c49c6ced25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:13:54.346541 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 601/1202] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:13:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 17 19:13:54.370576 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0cd3cb-b5ee-4110-9402-a0a75e56d0ef None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:13:54.467973 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0cd3cb-b5ee-4110-9402-a0a75e56d0ef None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:13:54.500205 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb74e366-8ca5-4e49-8215-b6e09a2dc393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:13:54.500989 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb74e366-8ca5-4e49-8215-b6e09a2dc393 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:13:54.501268 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb74e366-8ca5-4e49-8215-b6e09a2dc393 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:13:54.501540 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb74e366-8ca5-4e49-8215-b6e09a2dc393 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:13:54.511765 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb74e366-8ca5-4e49-8215-b6e09a2dc393 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:13:54.513141 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb74e366-8ca5-4e49-8215-b6e09a2dc393 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:13:54.530549 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-fb74e366-8ca5-4e49-8215-b6e09a2dc393 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:13:54.530549 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:13:54.530549 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:13:54.530549 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:13:54.530549 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:13:54.530549 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:13:54.530549 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:13:54.530549 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:13:54.530549 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:13:54.530549 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:13:54.530549 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:13:54.530549 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:13:54.530549 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 17 19:13:54.530549 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 17 19:13:54.530549 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 17 19:13:54.533439 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 17 19:13:54.533439 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:13:54.533439 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:13:54.533439 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:13:54.533439 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:13:54.533439 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:13:54.533439 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:13:54.533439 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:13:54.533439 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:13:54.533439 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:13:54.533439 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:13:54.533439 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:13:54.533439 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:13:54.533439 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:13:54.533439 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:13:54.533439 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:13:54.533439 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:13:54.534628 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 17 19:13:54.534628 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 17 19:13:54.534628 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:13:54.534628 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:13:54.534628 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 602/1203] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:13:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:13:54.537786 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2441a68-7d23-44a7-be72-81c49c6ced25 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:13:54.630444 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2441a68-7d23-44a7-be72-81c49c6ced25 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:13:54.662246 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b598ff59-b6f7-4375-8c20-d42bc971eb79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:13:54.662887 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b598ff59-b6f7-4375-8c20-d42bc971eb79 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:13:54.663165 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b598ff59-b6f7-4375-8c20-d42bc971eb79 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:13:54.663433 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b598ff59-b6f7-4375-8c20-d42bc971eb79 None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:13:54.664168 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b598ff59-b6f7-4375-8c20-d42bc971eb79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:13:54.664471 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b598ff59-b6f7-4375-8c20-d42bc971eb79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:13:54.665822 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b598ff59-b6f7-4375-8c20-d42bc971eb79 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:13:54.676477 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 602/1204] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 17 19:13:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:13:54.682157 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb74e366-8ca5-4e49-8215-b6e09a2dc393 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:13:54.778090 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb74e366-8ca5-4e49-8215-b6e09a2dc393 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:13:54.812533 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7dd492-31b1-47af-a03e-c17e98f27e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:13:54.813209 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7dd492-31b1-47af-a03e-c17e98f27e5c None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:13:54.813486 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7dd492-31b1-47af-a03e-c17e98f27e5c None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:13:54.813757 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7dd492-31b1-47af-a03e-c17e98f27e5c None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:13:54.814442 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7dd492-31b1-47af-a03e-c17e98f27e5c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:13:54.815892 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7dd492-31b1-47af-a03e-c17e98f27e5c None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:13:54.922402 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 603/1205] 10.222.0.58 () {60 vars in 1132 bytes} [Tue May 17 19:13:54 2022] POST /identity/v3/users => generated 251 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:13:57.132319 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743c7214-af9c-48ef-bb28-8ddd0b3c6ed9 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:13:57.132847 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743c7214-af9c-48ef-bb28-8ddd0b3c6ed9 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:13:57.132847 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743c7214-af9c-48ef-bb28-8ddd0b3c6ed9 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:13:57.134305 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 603/1206] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:13:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:13:57.139568 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab991b9-e4d1-42ae-8d65-a1d2a15c6aa0 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:13:57.139979 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab991b9-e4d1-42ae-8d65-a1d2a15c6aa0 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:13:57.140120 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab991b9-e4d1-42ae-8d65-a1d2a15c6aa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:13:57.156868 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-5ab991b9-e4d1-42ae-8d65-a1d2a15c6aa0 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:13:57.158052 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-5ab991b9-e4d1-42ae-8d65-a1d2a15c6aa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:13:57.221520 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 604/1207] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:13:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 17 19:13:57.244276 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743c7214-af9c-48ef-bb28-8ddd0b3c6ed9 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:13:57.340124 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743c7214-af9c-48ef-bb28-8ddd0b3c6ed9 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:13:57.371967 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035c7f18-b624-4e67-8058-7eb145803e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:13:57.372767 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035c7f18-b624-4e67-8058-7eb145803e62 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:13:57.373043 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035c7f18-b624-4e67-8058-7eb145803e62 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:13:57.373308 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035c7f18-b624-4e67-8058-7eb145803e62 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:13:57.383273 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035c7f18-b624-4e67-8058-7eb145803e62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 17 19:13:57.391329 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-035c7f18-b624-4e67-8058-7eb145803e62 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 17 19:13:57.391329 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:13:57.391329 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:13:57.391329 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:13:57.391329 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:13:57.391329 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:13:57.391329 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:13:57.391329 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:13:57.391329 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:13:57.391329 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:13:57.391329 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:13:57.391329 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:13:57.391329 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 17 19:13:57.391329 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) May 17 19:13:57.391329 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 17 19:13:57.391329 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 17 19:13:57.392530 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 17 19:13:57.392530 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 17 19:13:57.392530 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 17 19:13:57.392530 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 17 19:13:57.392530 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:13:57.392530 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:13:57.392530 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:13:57.392530 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:13:57.392530 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:13:57.392530 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:13:57.392530 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:13:57.392530 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:13:57.392530 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:13:57.392530 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:13:57.392530 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:13:57.392530 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:13:57.393793 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:13:57.393793 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:13:57.393793 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:13:57.393793 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:13:57.393793 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 17 19:13:57.393793 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 17 19:13:57.393793 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 17 19:13:57.393793 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 17 19:13:57.393793 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 17 19:13:57.393793 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 17 19:13:57.393793 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 17 19:13:57.393793 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:13:57.393793 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 604/1208] 10.222.0.58 () {56 vars in 1102 bytes} [Tue May 17 19:13:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:13:57.398559 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab991b9-e4d1-42ae-8d65-a1d2a15c6aa0 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:13:57.493589 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab991b9-e4d1-42ae-8d65-a1d2a15c6aa0 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:13:57.525750 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3b0769-63f6-4f13-bf7e-1f06b063cc57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:13:57.527046 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3b0769-63f6-4f13-bf7e-1f06b063cc57 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:13:57.527321 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3b0769-63f6-4f13-bf7e-1f06b063cc57 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:13:57.527620 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3b0769-63f6-4f13-bf7e-1f06b063cc57 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:13:57.528360 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3b0769-63f6-4f13-bf7e-1f06b063cc57 None admin] RBAC: Adding query filter params (name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:13:57.528663 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3b0769-63f6-4f13-bf7e-1f06b063cc57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:13:57.530103 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3b0769-63f6-4f13-bf7e-1f06b063cc57 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:13:57.537789 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 605/1209] 10.222.0.58 () {56 vars in 1103 bytes} [Tue May 17 19:13:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:13:57.543584 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035c7f18-b624-4e67-8058-7eb145803e62 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:13:57.638708 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035c7f18-b624-4e67-8058-7eb145803e62 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:13:57.669906 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48152c0f-cfe1-4aec-9b83-dded1539a74d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:13:57.670671 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48152c0f-cfe1-4aec-9b83-dded1539a74d None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:13:57.670944 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48152c0f-cfe1-4aec-9b83-dded1539a74d None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:13:57.671210 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48152c0f-cfe1-4aec-9b83-dded1539a74d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:13:57.680718 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48152c0f-cfe1-4aec-9b83-dded1539a74d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:13:57.682135 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48152c0f-cfe1-4aec-9b83-dded1539a74d None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:13:57.699188 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-48152c0f-cfe1-4aec-9b83-dded1539a74d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:13:57.699188 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:13:57.699188 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:13:57.699188 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:13:57.699188 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:13:57.699188 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:13:57.699188 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:13:57.699188 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:13:57.699188 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:13:57.699188 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:13:57.699188 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:13:57.699188 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:13:57.699188 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 17 19:13:57.699188 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 17 19:13:57.699188 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 17 19:13:57.700353 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 17 19:13:57.700353 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:13:57.700353 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:13:57.700353 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:13:57.700353 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:13:57.700353 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:13:57.700353 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:13:57.700353 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:13:57.700353 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:13:57.700353 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:13:57.700353 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:13:57.700353 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:13:57.700353 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:13:57.700353 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:13:57.700353 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:13:57.700353 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:13:57.700353 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:13:57.701310 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 17 19:13:57.701310 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 17 19:13:57.701310 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:13:57.701310 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:13:57.701310 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 605/1210] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:13:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:13:57.706111 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3b0769-63f6-4f13-bf7e-1f06b063cc57 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:13:57.802176 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3b0769-63f6-4f13-bf7e-1f06b063cc57 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:13:57.833391 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aeb9a2-997f-42ff-b662-5e461dd1ecdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:13:57.834040 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aeb9a2-997f-42ff-b662-5e461dd1ecdb None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:13:57.834317 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aeb9a2-997f-42ff-b662-5e461dd1ecdb None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:13:57.834586 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aeb9a2-997f-42ff-b662-5e461dd1ecdb None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:13:57.835286 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57aeb9a2-997f-42ff-b662-5e461dd1ecdb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:13:57.835617 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57aeb9a2-997f-42ff-b662-5e461dd1ecdb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:13:57.836981 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57aeb9a2-997f-42ff-b662-5e461dd1ecdb None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:13:57.853558 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 606/1211] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 17 19:13:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:13:57.859329 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48152c0f-cfe1-4aec-9b83-dded1539a74d None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:13:57.954073 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48152c0f-cfe1-4aec-9b83-dded1539a74d None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:13:57.986974 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb3f7b7-74ca-4824-9997-911eca36b1e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:13:57.987804 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb3f7b7-74ca-4824-9997-911eca36b1e8 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:13:57.988087 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb3f7b7-74ca-4824-9997-911eca36b1e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:13:57.988355 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb3f7b7-74ca-4824-9997-911eca36b1e8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:13:57.995842 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb3f7b7-74ca-4824-9997-911eca36b1e8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:13:57.997274 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb3f7b7-74ca-4824-9997-911eca36b1e8 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:13:58.011837 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-ecb3f7b7-74ca-4824-9997-911eca36b1e8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 17 19:13:58.011837 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:13:58.011837 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:13:58.011837 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:13:58.011837 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:13:58.011837 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:13:58.011837 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:13:58.011837 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:13:58.011837 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:13:58.011837 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:13:58.011837 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:13:58.011837 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:13:58.011837 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 17 19:13:58.011837 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_user(user_id) May 17 19:13:58.011837 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 17 19:13:58.011837 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 17 19:13:58.013012 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:13:58.013012 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:13:58.013012 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 17 19:13:58.013012 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 17 19:13:58.013012 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 17 19:13:58.013012 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 17 19:13:58.013012 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:13:58.013012 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:13:58.013012 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:13:58.013012 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:13:58.013012 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:13:58.013012 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:13:58.013012 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:13:58.013012 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:13:58.013012 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:13:58.013012 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:13:58.013012 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:13:58.013012 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:13:58.014037 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:13:58.014037 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:13:58.014037 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 17 19:13:58.014037 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 17 19:13:58.014037 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 17 19:13:58.014037 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 17 19:13:58.014037 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 17 19:13:58.014037 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 17 19:13:58.014037 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 17 19:13:58.014037 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:13:58.014037 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 606/1212] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:13:57 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:13:58.018986 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aeb9a2-997f-42ff-b662-5e461dd1ecdb None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:13:58.119255 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aeb9a2-997f-42ff-b662-5e461dd1ecdb None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:13:58.153973 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed211fe2-ac84-45bb-894b-9e1b8599e84a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:13:58.154614 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed211fe2-ac84-45bb-894b-9e1b8599e84a None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:13:58.154899 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed211fe2-ac84-45bb-894b-9e1b8599e84a None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:13:58.155181 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed211fe2-ac84-45bb-894b-9e1b8599e84a None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:13:58.156040 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed211fe2-ac84-45bb-894b-9e1b8599e84a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:13:58.156344 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed211fe2-ac84-45bb-894b-9e1b8599e84a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:13:58.157783 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed211fe2-ac84-45bb-894b-9e1b8599e84a None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:13:58.176945 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 607/1213] 10.222.0.58 () {56 vars in 1143 bytes} [Tue May 17 19:13:58 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:13:58.183247 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb3f7b7-74ca-4824-9997-911eca36b1e8 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:13:58.284850 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb3f7b7-74ca-4824-9997-911eca36b1e8 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:13:58.317589 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b59691-1095-4458-8e93-18442d1a3b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:13:58.318264 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b59691-1095-4458-8e93-18442d1a3b7a None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:13:58.318544 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b59691-1095-4458-8e93-18442d1a3b7a None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:13:58.318819 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b59691-1095-4458-8e93-18442d1a3b7a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:13:58.382532 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b59691-1095-4458-8e93-18442d1a3b7a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:13:58.383931 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b59691-1095-4458-8e93-18442d1a3b7a None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:13:58.475056 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 607/1214] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 17 19:13:58 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 17 19:13:58.480792 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed211fe2-ac84-45bb-894b-9e1b8599e84a None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:13:58.575229 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed211fe2-ac84-45bb-894b-9e1b8599e84a None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:13:58.607118 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2d7d37-92c2-4063-99d6-b9e05dde1edb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:13:58.607925 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2d7d37-92c2-4063-99d6-b9e05dde1edb None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:13:58.608199 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2d7d37-92c2-4063-99d6-b9e05dde1edb None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:13:58.608464 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2d7d37-92c2-4063-99d6-b9e05dde1edb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:13:58.617974 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2d7d37-92c2-4063-99d6-b9e05dde1edb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:13:58.619415 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2d7d37-92c2-4063-99d6-b9e05dde1edb None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:13:58.637199 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-1d2d7d37-92c2-4063-99d6-b9e05dde1edb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:13:58.637199 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:13:58.637199 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:13:58.637199 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:13:58.637199 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:13:58.637199 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:13:58.637199 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:13:58.637199 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:13:58.637199 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:13:58.637199 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:13:58.637199 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:13:58.637199 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:13:58.637199 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 17 19:13:58.637199 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 17 19:13:58.637199 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 17 19:13:58.638182 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 17 19:13:58.638182 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:13:58.638182 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:13:58.638182 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:13:58.638182 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:13:58.638182 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:13:58.638182 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:13:58.638182 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:13:58.638182 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:13:58.638182 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:13:58.638182 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:13:58.638182 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:13:58.638182 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:13:58.638182 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:13:58.638182 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:13:58.638182 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:13:58.638182 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:13:58.639170 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 17 19:13:58.639170 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 17 19:13:58.639170 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:13:58.639170 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:13:58.639170 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 608/1215] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:13:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:13:58.644055 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b59691-1095-4458-8e93-18442d1a3b7a None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:13:58.737241 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b59691-1095-4458-8e93-18442d1a3b7a None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:13:58.769197 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab562e9-7bc4-4cd1-a08f-cd6a2de3d197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:13:58.769848 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab562e9-7bc4-4cd1-a08f-cd6a2de3d197 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:13:58.770134 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab562e9-7bc4-4cd1-a08f-cd6a2de3d197 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:13:58.770404 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab562e9-7bc4-4cd1-a08f-cd6a2de3d197 None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:13:58.771111 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab562e9-7bc4-4cd1-a08f-cd6a2de3d197 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:13:58.771412 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab562e9-7bc4-4cd1-a08f-cd6a2de3d197 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:13:58.772799 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab562e9-7bc4-4cd1-a08f-cd6a2de3d197 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:13:58.783759 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 608/1216] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 17 19:13:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:13:58.789392 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2d7d37-92c2-4063-99d6-b9e05dde1edb None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:13:58.885146 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2d7d37-92c2-4063-99d6-b9e05dde1edb None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:13:58.918369 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ff530e-4fbc-4422-906f-5e40f25212cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:13:58.919090 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ff530e-4fbc-4422-906f-5e40f25212cc None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:13:58.919370 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ff530e-4fbc-4422-906f-5e40f25212cc None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:13:58.919682 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ff530e-4fbc-4422-906f-5e40f25212cc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:13:58.926306 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ff530e-4fbc-4422-906f-5e40f25212cc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:13:58.927821 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ff530e-4fbc-4422-906f-5e40f25212cc None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:13:58.941982 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-a5ff530e-4fbc-4422-906f-5e40f25212cc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 17 19:13:58.941982 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:13:58.941982 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:13:58.941982 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:13:58.941982 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:13:58.941982 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:13:58.941982 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:13:58.941982 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:13:58.941982 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:13:58.941982 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:13:58.941982 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:13:58.941982 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:13:58.941982 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 17 19:13:58.941982 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(project_id) May 17 19:13:58.941982 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 17 19:13:58.942890 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 17 19:13:58.942890 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:13:58.942890 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:13:58.942890 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:13:58.942890 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:13:58.942890 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:13:58.942890 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:13:58.942890 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:13:58.942890 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:13:58.942890 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:13:58.942890 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:13:58.942890 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:13:58.942890 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:13:58.942890 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:13:58.942890 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:13:58.942890 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:13:58.942890 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:13:58.943868 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 17 19:13:58.943868 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 17 19:13:58.943868 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 17 19:13:58.943868 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 17 19:13:58.943868 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 17 19:13:58.943868 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 17 19:13:58.943868 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 17 19:13:58.943868 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:13:58.947642 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 609/1217] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 17 19:13:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:13:58.948968 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab562e9-7bc4-4cd1-a08f-cd6a2de3d197 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:13:59.044703 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab562e9-7bc4-4cd1-a08f-cd6a2de3d197 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:13:59.077535 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb84b75-e577-4354-87ac-f7cb7c7d9ffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:13:59.078174 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb84b75-e577-4354-87ac-f7cb7c7d9ffa None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:13:59.078448 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb84b75-e577-4354-87ac-f7cb7c7d9ffa None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:13:59.078722 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb84b75-e577-4354-87ac-f7cb7c7d9ffa None admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:13:59.079509 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb84b75-e577-4354-87ac-f7cb7c7d9ffa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:13:59.079811 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb84b75-e577-4354-87ac-f7cb7c7d9ffa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:13:59.081222 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb84b75-e577-4354-87ac-f7cb7c7d9ffa None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:13:59.093213 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 609/1218] 10.222.0.58 () {56 vars in 1148 bytes} [Tue May 17 19:13:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:13:59.099420 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ff530e-4fbc-4422-906f-5e40f25212cc None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:13:59.194521 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ff530e-4fbc-4422-906f-5e40f25212cc None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:13:59.226863 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b243b54-7507-4341-aa92-c9ff1de34670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:13:59.227527 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b243b54-7507-4341-aa92-c9ff1de34670 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:13:59.227831 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b243b54-7507-4341-aa92-c9ff1de34670 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:13:59.228106 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b243b54-7507-4341-aa92-c9ff1de34670 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:13:59.228924 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b243b54-7507-4341-aa92-c9ff1de34670 None admin] RBAC: Adding query filter params (role.id=1754a95b778c4e20b53b20cb52ada92b, scope.project.id=1a9d3a0631284c9fb113266036993e2d, user.id=06538aab5f194af7b7c677d6b94aef2a) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:13:59.229260 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b243b54-7507-4341-aa92-c9ff1de34670 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:13:59.230706 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b243b54-7507-4341-aa92-c9ff1de34670 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:13:59.236027 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 610/1219] 10.222.0.58 () {56 vars in 1374 bytes} [Tue May 17 19:13:59 2022] GET /identity/v3/role_assignments?user.id=06538aab5f194af7b7c677d6b94aef2a&scope.project.id=1a9d3a0631284c9fb113266036993e2d&role.id=1754a95b778c4e20b53b20cb52ada92b => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:14:01.370698 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0721c4-04d0-4097-abfa-ce036926ccd1 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:14:01.371215 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0721c4-04d0-4097-abfa-ce036926ccd1 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:14:01.371215 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0721c4-04d0-4097-abfa-ce036926ccd1 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:14:01.372526 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 610/1220] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:14:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:14:01.377640 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabe5bd3-3adc-489d-aaa5-934b21262764 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:14:01.378050 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabe5bd3-3adc-489d-aaa5-934b21262764 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:14:01.378191 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabe5bd3-3adc-489d-aaa5-934b21262764 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:14:01.395782 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-cabe5bd3-3adc-489d-aaa5-934b21262764 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:14:01.396946 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-cabe5bd3-3adc-489d-aaa5-934b21262764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:14:01.460492 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 611/1221] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:14:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 17 19:14:01.482679 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0721c4-04d0-4097-abfa-ce036926ccd1 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:14:01.579067 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0721c4-04d0-4097-abfa-ce036926ccd1 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:14:01.611218 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2c6abe-9ce3-4e4e-bda5-22dd27a77a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:14:01.612107 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2c6abe-9ce3-4e4e-bda5-22dd27a77a04 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:14:01.612382 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2c6abe-9ce3-4e4e-bda5-22dd27a77a04 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:14:01.612650 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2c6abe-9ce3-4e4e-bda5-22dd27a77a04 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:14:01.623011 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2c6abe-9ce3-4e4e-bda5-22dd27a77a04 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 17 19:14:01.630754 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-9b2c6abe-9ce3-4e4e-bda5-22dd27a77a04 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 17 19:14:01.630754 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:14:01.630754 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:14:01.630754 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:14:01.630754 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:14:01.630754 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:14:01.630754 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:14:01.630754 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:14:01.630754 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:14:01.630754 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:14:01.630754 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:14:01.630754 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:14:01.630754 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 17 19:14:01.630754 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) May 17 19:14:01.630754 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 17 19:14:01.630754 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 17 19:14:01.631861 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 17 19:14:01.631861 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 17 19:14:01.631861 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 17 19:14:01.631861 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 17 19:14:01.631861 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:14:01.631861 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:14:01.631861 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:14:01.631861 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:14:01.631861 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:14:01.631861 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:14:01.631861 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:14:01.631861 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:14:01.631861 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:14:01.631861 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:14:01.631861 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:14:01.631861 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:14:01.633124 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:14:01.633124 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:14:01.633124 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:14:01.633124 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:14:01.633124 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 17 19:14:01.633124 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 17 19:14:01.633124 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 17 19:14:01.633124 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 17 19:14:01.633124 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 17 19:14:01.633124 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 17 19:14:01.633124 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 17 19:14:01.633124 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:14:01.633124 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 611/1222] 10.222.0.58 () {56 vars in 1102 bytes} [Tue May 17 19:14:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:14:01.638409 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabe5bd3-3adc-489d-aaa5-934b21262764 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:14:01.732062 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabe5bd3-3adc-489d-aaa5-934b21262764 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:14:01.764389 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6656a968-945f-4efc-a5ef-ee571fc95e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:14:01.767445 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6656a968-945f-4efc-a5ef-ee571fc95e19 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:14:01.767445 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6656a968-945f-4efc-a5ef-ee571fc95e19 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:14:01.767445 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6656a968-945f-4efc-a5ef-ee571fc95e19 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:14:01.767445 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6656a968-945f-4efc-a5ef-ee571fc95e19 None admin] RBAC: Adding query filter params (name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:14:01.767445 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6656a968-945f-4efc-a5ef-ee571fc95e19 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:14:01.768071 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6656a968-945f-4efc-a5ef-ee571fc95e19 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:14:01.775860 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 612/1223] 10.222.0.58 () {56 vars in 1103 bytes} [Tue May 17 19:14:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:14:01.781587 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2c6abe-9ce3-4e4e-bda5-22dd27a77a04 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:14:01.875675 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2c6abe-9ce3-4e4e-bda5-22dd27a77a04 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:14:01.906580 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942345de-76b1-4167-a878-ec23e37d8820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:14:01.907347 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942345de-76b1-4167-a878-ec23e37d8820 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:14:01.907652 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942345de-76b1-4167-a878-ec23e37d8820 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:14:01.907918 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942345de-76b1-4167-a878-ec23e37d8820 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:14:01.918296 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942345de-76b1-4167-a878-ec23e37d8820 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:14:01.919737 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942345de-76b1-4167-a878-ec23e37d8820 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:14:01.936275 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-942345de-76b1-4167-a878-ec23e37d8820 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:14:01.936275 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:14:01.936275 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:14:01.936275 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:14:01.936275 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:14:01.936275 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:14:01.936275 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:14:01.936275 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:14:01.936275 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:14:01.936275 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:14:01.936275 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:14:01.936275 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:14:01.936275 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 17 19:14:01.936275 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 17 19:14:01.936275 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 17 19:14:01.937346 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 17 19:14:01.937346 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:14:01.937346 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:14:01.937346 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:14:01.937346 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:14:01.937346 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:14:01.937346 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:14:01.937346 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:14:01.937346 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:14:01.937346 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:14:01.937346 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:14:01.937346 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:14:01.937346 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:14:01.937346 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:14:01.937346 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:14:01.937346 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:14:01.937346 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:14:01.938634 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 17 19:14:01.938634 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 17 19:14:01.938634 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:14:01.938634 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:14:01.938634 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 612/1224] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:14:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:14:01.943062 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6656a968-945f-4efc-a5ef-ee571fc95e19 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:14:02.038577 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6656a968-945f-4efc-a5ef-ee571fc95e19 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:14:02.070697 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b730f8d9-9356-4edc-8aad-324e9408368e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:14:02.071348 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b730f8d9-9356-4edc-8aad-324e9408368e None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:14:02.071663 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b730f8d9-9356-4edc-8aad-324e9408368e None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:14:02.071934 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b730f8d9-9356-4edc-8aad-324e9408368e None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:14:02.072641 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b730f8d9-9356-4edc-8aad-324e9408368e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:14:02.072937 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b730f8d9-9356-4edc-8aad-324e9408368e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:14:02.074288 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b730f8d9-9356-4edc-8aad-324e9408368e None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:14:02.086228 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 613/1225] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 17 19:14:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:14:02.092336 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942345de-76b1-4167-a878-ec23e37d8820 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:14:02.187895 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942345de-76b1-4167-a878-ec23e37d8820 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:14:02.220429 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2d63f7-e13a-4b64-9ca4-4c4be7ea71ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:14:02.222507 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2d63f7-e13a-4b64-9ca4-4c4be7ea71ec None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:14:02.222507 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2d63f7-e13a-4b64-9ca4-4c4be7ea71ec None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:14:02.222507 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2d63f7-e13a-4b64-9ca4-4c4be7ea71ec None admin] PATH_INFO: `/v3/users/placement` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:14:02.228919 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2d63f7-e13a-4b64-9ca4-4c4be7ea71ec None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:14:02.230882 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2d63f7-e13a-4b64-9ca4-4c4be7ea71ec None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:14:02.244610 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-da2d63f7-e13a-4b64-9ca4-4c4be7ea71ec None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 17 19:14:02.244610 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:14:02.244610 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:14:02.244610 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:14:02.244610 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:14:02.244610 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:14:02.244610 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:14:02.244610 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:14:02.244610 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:14:02.244610 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:14:02.244610 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:14:02.244610 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:14:02.244610 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 17 19:14:02.244610 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_user(user_id) May 17 19:14:02.244610 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 17 19:14:02.244610 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 17 19:14:02.245701 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:14:02.245701 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:14:02.245701 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 17 19:14:02.245701 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 17 19:14:02.245701 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 17 19:14:02.245701 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 17 19:14:02.245701 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:14:02.245701 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:14:02.245701 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:14:02.245701 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:14:02.245701 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:14:02.245701 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:14:02.245701 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:14:02.245701 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:14:02.245701 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:14:02.245701 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:14:02.245701 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:14:02.245701 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:14:02.246972 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:14:02.246972 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:14:02.246972 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 17 19:14:02.246972 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 17 19:14:02.246972 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 17 19:14:02.246972 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 17 19:14:02.246972 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 17 19:14:02.246972 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 17 19:14:02.246972 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 17 19:14:02.246972 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:14:02.246972 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 613/1226] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:14:02 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:14:02.251409 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b730f8d9-9356-4edc-8aad-324e9408368e None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:14:02.346725 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b730f8d9-9356-4edc-8aad-324e9408368e None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:14:02.379168 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da00721d-e1ae-4dbb-893d-8e1dcd8e944e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:14:02.379851 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da00721d-e1ae-4dbb-893d-8e1dcd8e944e None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:14:02.380129 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da00721d-e1ae-4dbb-893d-8e1dcd8e944e None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:14:02.380398 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da00721d-e1ae-4dbb-893d-8e1dcd8e944e None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:14:02.381218 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da00721d-e1ae-4dbb-893d-8e1dcd8e944e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:14:02.381522 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da00721d-e1ae-4dbb-893d-8e1dcd8e944e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:14:02.382938 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da00721d-e1ae-4dbb-893d-8e1dcd8e944e None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:14:02.399145 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 614/1227] 10.222.0.58 () {56 vars in 1143 bytes} [Tue May 17 19:14:02 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:14:02.405317 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2d63f7-e13a-4b64-9ca4-4c4be7ea71ec None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:14:02.499890 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2d63f7-e13a-4b64-9ca4-4c4be7ea71ec None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:14:02.532490 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a4d3ed-36d7-4844-a986-1be292a7badc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:14:02.538065 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a4d3ed-36d7-4844-a986-1be292a7badc None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:14:02.538065 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a4d3ed-36d7-4844-a986-1be292a7badc None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:14:02.538065 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a4d3ed-36d7-4844-a986-1be292a7badc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:14:02.543961 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a4d3ed-36d7-4844-a986-1be292a7badc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:14:02.545399 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a4d3ed-36d7-4844-a986-1be292a7badc None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:14:02.561584 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-71a4d3ed-36d7-4844-a986-1be292a7badc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:14:02.561584 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:14:02.561584 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:14:02.561584 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:14:02.561584 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:14:02.561584 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:14:02.561584 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:14:02.561584 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:14:02.561584 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:14:02.561584 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:14:02.561584 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:14:02.561584 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:14:02.561584 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 17 19:14:02.561584 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 17 19:14:02.561584 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 17 19:14:02.562620 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 17 19:14:02.562620 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:14:02.562620 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:14:02.562620 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:14:02.562620 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:14:02.562620 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:14:02.562620 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:14:02.562620 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:14:02.562620 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:14:02.562620 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:14:02.562620 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:14:02.562620 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:14:02.562620 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:14:02.562620 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:14:02.562620 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:14:02.562620 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:14:02.562620 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:14:02.563764 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 17 19:14:02.563764 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 17 19:14:02.563764 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:14:02.563764 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:14:02.563764 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 614/1228] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:14:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:14:02.568433 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da00721d-e1ae-4dbb-893d-8e1dcd8e944e None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:14:02.664281 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da00721d-e1ae-4dbb-893d-8e1dcd8e944e None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:14:02.695726 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761fdcf8-015c-4553-a5bf-c19e65070a56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:14:02.696367 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761fdcf8-015c-4553-a5bf-c19e65070a56 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:14:02.696643 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761fdcf8-015c-4553-a5bf-c19e65070a56 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:14:02.696914 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761fdcf8-015c-4553-a5bf-c19e65070a56 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:14:02.697617 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761fdcf8-015c-4553-a5bf-c19e65070a56 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:14:02.698481 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761fdcf8-015c-4553-a5bf-c19e65070a56 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:14:02.699903 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761fdcf8-015c-4553-a5bf-c19e65070a56 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:14:02.710662 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 615/1229] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 17 19:14:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:14:02.716788 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a4d3ed-36d7-4844-a986-1be292a7badc None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:14:02.812994 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a4d3ed-36d7-4844-a986-1be292a7badc None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:14:02.844429 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bfd2ff-777a-4474-b5df-6cf1b62a1a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:14:02.845547 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bfd2ff-777a-4474-b5df-6cf1b62a1a6d None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:14:02.846780 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bfd2ff-777a-4474-b5df-6cf1b62a1a6d None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:14:02.847054 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bfd2ff-777a-4474-b5df-6cf1b62a1a6d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:14:02.856088 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bfd2ff-777a-4474-b5df-6cf1b62a1a6d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:14:02.857526 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bfd2ff-777a-4474-b5df-6cf1b62a1a6d None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:14:02.871988 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-f6bfd2ff-777a-4474-b5df-6cf1b62a1a6d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 17 19:14:02.871988 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:14:02.871988 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:14:02.871988 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:14:02.871988 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:14:02.871988 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:14:02.871988 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:14:02.871988 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:14:02.871988 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:14:02.871988 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:14:02.871988 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:14:02.871988 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:14:02.871988 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 17 19:14:02.871988 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(project_id) May 17 19:14:02.871988 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 17 19:14:02.873054 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 17 19:14:02.873054 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:14:02.873054 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:14:02.873054 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:14:02.873054 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:14:02.873054 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:14:02.873054 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:14:02.873054 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:14:02.873054 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:14:02.873054 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:14:02.873054 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:14:02.873054 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:14:02.873054 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:14:02.873054 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:14:02.873054 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:14:02.873054 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:14:02.873054 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:14:02.874191 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 17 19:14:02.874191 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 17 19:14:02.874191 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 17 19:14:02.874191 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 17 19:14:02.874191 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 17 19:14:02.874191 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 17 19:14:02.874191 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 17 19:14:02.874191 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:14:02.874191 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 615/1230] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 17 19:14:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:14:02.878997 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761fdcf8-015c-4553-a5bf-c19e65070a56 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:14:02.974321 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761fdcf8-015c-4553-a5bf-c19e65070a56 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:14:03.006085 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7208366a-78c2-4ce2-a6b4-b65f6f0ee65c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:14:03.006732 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7208366a-78c2-4ce2-a6b4-b65f6f0ee65c None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:14:03.007009 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7208366a-78c2-4ce2-a6b4-b65f6f0ee65c None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:14:03.007280 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7208366a-78c2-4ce2-a6b4-b65f6f0ee65c None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:14:03.008083 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7208366a-78c2-4ce2-a6b4-b65f6f0ee65c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:14:03.008388 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7208366a-78c2-4ce2-a6b4-b65f6f0ee65c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:14:03.009931 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7208366a-78c2-4ce2-a6b4-b65f6f0ee65c None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:14:03.021970 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 616/1231] 10.222.0.58 () {56 vars in 1148 bytes} [Tue May 17 19:14:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:14:03.028006 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bfd2ff-777a-4474-b5df-6cf1b62a1a6d None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:14:03.124205 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bfd2ff-777a-4474-b5df-6cf1b62a1a6d None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:14:03.156980 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cb79ea-af7c-4b9e-8d25-3a3615af3508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:14:03.161701 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cb79ea-af7c-4b9e-8d25-3a3615af3508 None admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:14:03.161701 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cb79ea-af7c-4b9e-8d25-3a3615af3508 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:14:03.161701 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cb79ea-af7c-4b9e-8d25-3a3615af3508 None admin] PATH_INFO: `/v3/projects/1a9d3a0631284c9fb113266036993e2d/users/06538aab5f194af7b7c677d6b94aef2a/roles/1754a95b778c4e20b53b20cb52ada92b` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:14:03.172586 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cb79ea-af7c-4b9e-8d25-3a3615af3508 None admin] RBAC: Authorizing `identity:create_grant(project_id=1a9d3a0631284c9fb113266036993e2d, user_id=06538aab5f194af7b7c677d6b94aef2a, role_id=1754a95b778c4e20b53b20cb52ada92b)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:14:03.174037 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cb79ea-af7c-4b9e-8d25-3a3615af3508 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:14:03.242134 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 616/1232] 10.222.0.58 () {58 vars in 1439 bytes} [Tue May 17 19:14:03 2022] PUT /identity/v3/projects/1a9d3a0631284c9fb113266036993e2d/users/06538aab5f194af7b7c677d6b94aef2a/roles/1754a95b778c4e20b53b20cb52ada92b => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:14:05.409709 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460ba9bb-242f-44bf-984e-799592b3eb57 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:14:05.410214 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460ba9bb-242f-44bf-984e-799592b3eb57 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:14:05.410214 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460ba9bb-242f-44bf-984e-799592b3eb57 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:14:05.411560 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 617/1233] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:14:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:14:05.417324 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be6444c-4013-4762-bab4-e80bd7b225a7 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:14:05.417748 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be6444c-4013-4762-bab4-e80bd7b225a7 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:14:05.417870 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be6444c-4013-4762-bab4-e80bd7b225a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:14:05.434152 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-6be6444c-4013-4762-bab4-e80bd7b225a7 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:14:05.435325 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-6be6444c-4013-4762-bab4-e80bd7b225a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:14:05.498296 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 617/1234] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:14:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 17 19:14:05.521130 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460ba9bb-242f-44bf-984e-799592b3eb57 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:14:05.618905 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460ba9bb-242f-44bf-984e-799592b3eb57 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:14:05.651819 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eb201e-c79e-40fc-af54-fc2480c4b9c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:14:05.652605 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eb201e-c79e-40fc-af54-fc2480c4b9c6 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:14:05.652884 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eb201e-c79e-40fc-af54-fc2480c4b9c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:14:05.653154 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eb201e-c79e-40fc-af54-fc2480c4b9c6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:14:05.663074 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4eb201e-c79e-40fc-af54-fc2480c4b9c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 17 19:14:05.670697 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-c4eb201e-c79e-40fc-af54-fc2480c4b9c6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 17 19:14:05.670697 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:14:05.670697 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:14:05.670697 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:14:05.670697 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:14:05.670697 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:14:05.670697 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:14:05.670697 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:14:05.670697 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:14:05.670697 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:14:05.670697 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:14:05.670697 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:14:05.670697 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 17 19:14:05.670697 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) May 17 19:14:05.670697 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 17 19:14:05.670697 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 17 19:14:05.671910 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 17 19:14:05.671910 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 17 19:14:05.671910 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 17 19:14:05.671910 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 17 19:14:05.671910 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:14:05.671910 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:14:05.671910 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:14:05.671910 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:14:05.671910 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:14:05.671910 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:14:05.671910 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:14:05.671910 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:14:05.671910 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:14:05.671910 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:14:05.671910 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:14:05.671910 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:14:05.672972 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:14:05.672972 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:14:05.672972 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:14:05.672972 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:14:05.672972 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 17 19:14:05.672972 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 17 19:14:05.672972 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 17 19:14:05.672972 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 17 19:14:05.672972 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 17 19:14:05.672972 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 17 19:14:05.672972 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 17 19:14:05.672972 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:14:05.672972 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 618/1235] 10.222.0.58 () {56 vars in 1102 bytes} [Tue May 17 19:14:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:14:05.677886 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be6444c-4013-4762-bab4-e80bd7b225a7 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:14:05.772916 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be6444c-4013-4762-bab4-e80bd7b225a7 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:14:05.805044 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cd47de-d06e-4a08-a8c6-ea2bc00f19e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:14:05.805695 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cd47de-d06e-4a08-a8c6-ea2bc00f19e1 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:14:05.805971 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cd47de-d06e-4a08-a8c6-ea2bc00f19e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:14:05.806239 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cd47de-d06e-4a08-a8c6-ea2bc00f19e1 None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:14:05.807015 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cd47de-d06e-4a08-a8c6-ea2bc00f19e1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:14:05.807854 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cd47de-d06e-4a08-a8c6-ea2bc00f19e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:14:05.809213 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cd47de-d06e-4a08-a8c6-ea2bc00f19e1 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:14:05.817376 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 618/1236] 10.222.0.58 () {56 vars in 1103 bytes} [Tue May 17 19:14:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:14:05.823685 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eb201e-c79e-40fc-af54-fc2480c4b9c6 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:14:05.917489 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eb201e-c79e-40fc-af54-fc2480c4b9c6 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:14:05.950204 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87e6ef5-2ec2-40e1-b669-ee81003bd56b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:14:05.950983 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87e6ef5-2ec2-40e1-b669-ee81003bd56b None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:14:05.951258 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87e6ef5-2ec2-40e1-b669-ee81003bd56b None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:14:05.951554 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87e6ef5-2ec2-40e1-b669-ee81003bd56b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:14:05.961821 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87e6ef5-2ec2-40e1-b669-ee81003bd56b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:14:05.963243 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87e6ef5-2ec2-40e1-b669-ee81003bd56b None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:14:05.979997 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-b87e6ef5-2ec2-40e1-b669-ee81003bd56b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:14:05.979997 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:14:05.979997 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:14:05.979997 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:14:05.979997 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:14:05.979997 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:14:05.979997 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:14:05.979997 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:14:05.979997 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:14:05.979997 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:14:05.979997 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:14:05.979997 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:14:05.979997 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 17 19:14:05.979997 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 17 19:14:05.979997 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 17 19:14:05.981110 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 17 19:14:05.981110 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:14:05.981110 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:14:05.981110 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:14:05.981110 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:14:05.981110 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:14:05.981110 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:14:05.981110 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:14:05.981110 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:14:05.981110 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:14:05.981110 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:14:05.981110 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:14:05.981110 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:14:05.981110 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:14:05.981110 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:14:05.981110 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:14:05.981110 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:14:05.982386 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 17 19:14:05.982386 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 17 19:14:05.982386 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:14:05.982386 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:14:05.982386 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 619/1237] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:14:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:14:05.986756 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cd47de-d06e-4a08-a8c6-ea2bc00f19e1 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:14:06.080784 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cd47de-d06e-4a08-a8c6-ea2bc00f19e1 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:14:06.112291 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f72ced7-0fde-4e65-82ad-4eaa292ac83e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:14:06.112932 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f72ced7-0fde-4e65-82ad-4eaa292ac83e None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:14:06.113209 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f72ced7-0fde-4e65-82ad-4eaa292ac83e None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:14:06.113477 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f72ced7-0fde-4e65-82ad-4eaa292ac83e None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:14:06.114184 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f72ced7-0fde-4e65-82ad-4eaa292ac83e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:14:06.114484 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f72ced7-0fde-4e65-82ad-4eaa292ac83e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:14:06.115865 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f72ced7-0fde-4e65-82ad-4eaa292ac83e None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:14:06.127138 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 619/1238] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 17 19:14:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:14:06.132647 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87e6ef5-2ec2-40e1-b669-ee81003bd56b None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:14:06.228956 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87e6ef5-2ec2-40e1-b669-ee81003bd56b None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:14:06.261273 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a3c85f-817d-47ba-b750-024956194b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:14:06.262097 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a3c85f-817d-47ba-b750-024956194b9d None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:14:06.262373 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a3c85f-817d-47ba-b750-024956194b9d None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:14:06.262646 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a3c85f-817d-47ba-b750-024956194b9d None admin] PATH_INFO: `/v3/users/placement` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:14:06.269874 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a3c85f-817d-47ba-b750-024956194b9d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:14:06.271307 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a3c85f-817d-47ba-b750-024956194b9d None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:14:06.285419 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-f5a3c85f-817d-47ba-b750-024956194b9d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 17 19:14:06.285419 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:14:06.285419 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:14:06.285419 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:14:06.285419 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:14:06.285419 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:14:06.285419 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:14:06.285419 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:14:06.285419 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:14:06.285419 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:14:06.285419 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:14:06.285419 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:14:06.285419 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 17 19:14:06.285419 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_user(user_id) May 17 19:14:06.285419 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 17 19:14:06.285419 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 17 19:14:06.286538 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:14:06.286538 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:14:06.286538 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 17 19:14:06.286538 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 17 19:14:06.286538 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 17 19:14:06.286538 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 17 19:14:06.286538 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:14:06.286538 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:14:06.286538 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:14:06.286538 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:14:06.286538 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:14:06.286538 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:14:06.286538 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:14:06.286538 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:14:06.286538 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:14:06.286538 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:14:06.286538 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:14:06.286538 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:14:06.287697 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:14:06.287697 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:14:06.287697 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 17 19:14:06.287697 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 17 19:14:06.287697 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 17 19:14:06.287697 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 17 19:14:06.287697 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 17 19:14:06.287697 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 17 19:14:06.287697 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 17 19:14:06.287697 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:14:06.287697 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 620/1239] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:14:06 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:14:06.292886 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f72ced7-0fde-4e65-82ad-4eaa292ac83e None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:14:06.388167 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f72ced7-0fde-4e65-82ad-4eaa292ac83e None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:14:06.419364 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c2c5b7-3251-420c-8298-3ae3cbab55fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:14:06.420052 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c2c5b7-3251-420c-8298-3ae3cbab55fd None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:14:06.420322 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c2c5b7-3251-420c-8298-3ae3cbab55fd None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:14:06.420594 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c2c5b7-3251-420c-8298-3ae3cbab55fd None admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:14:06.421416 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c2c5b7-3251-420c-8298-3ae3cbab55fd None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:14:06.421722 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c2c5b7-3251-420c-8298-3ae3cbab55fd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:14:06.423164 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c2c5b7-3251-420c-8298-3ae3cbab55fd None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:14:06.439923 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 620/1240] 10.222.0.58 () {56 vars in 1143 bytes} [Tue May 17 19:14:06 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:14:06.445800 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a3c85f-817d-47ba-b750-024956194b9d None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:14:06.543833 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a3c85f-817d-47ba-b750-024956194b9d None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:14:06.575768 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bbbad1-5b96-45ed-9bd5-cbd9a1435e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:14:06.576404 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bbbad1-5b96-45ed-9bd5-cbd9a1435e26 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:14:06.576682 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bbbad1-5b96-45ed-9bd5-cbd9a1435e26 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:14:06.576954 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bbbad1-5b96-45ed-9bd5-cbd9a1435e26 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:14:06.640462 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bbbad1-5b96-45ed-9bd5-cbd9a1435e26 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:14:06.641907 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bbbad1-5b96-45ed-9bd5-cbd9a1435e26 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:14:06.736056 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 621/1241] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 17 19:14:06 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 17 19:14:06.741826 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c2c5b7-3251-420c-8298-3ae3cbab55fd None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:14:06.833299 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c2c5b7-3251-420c-8298-3ae3cbab55fd None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:14:06.866434 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4baf2f1-8967-45f0-80ce-96943fe5d6f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:14:06.867238 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4baf2f1-8967-45f0-80ce-96943fe5d6f9 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:14:06.867536 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4baf2f1-8967-45f0-80ce-96943fe5d6f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:14:06.867804 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4baf2f1-8967-45f0-80ce-96943fe5d6f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:14:06.877654 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4baf2f1-8967-45f0-80ce-96943fe5d6f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:14:06.879129 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4baf2f1-8967-45f0-80ce-96943fe5d6f9 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:14:06.895907 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-a4baf2f1-8967-45f0-80ce-96943fe5d6f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:14:06.895907 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:14:06.895907 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:14:06.895907 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:14:06.895907 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:14:06.895907 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:14:06.895907 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:14:06.895907 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:14:06.895907 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:14:06.895907 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:14:06.895907 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:14:06.895907 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:14:06.895907 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 17 19:14:06.895907 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 17 19:14:06.895907 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 17 19:14:06.896956 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 17 19:14:06.896956 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:14:06.896956 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:14:06.896956 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:14:06.896956 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:14:06.896956 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:14:06.896956 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:14:06.896956 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:14:06.896956 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:14:06.896956 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:14:06.896956 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:14:06.896956 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:14:06.896956 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:14:06.896956 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:14:06.896956 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:14:06.896956 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:14:06.896956 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:14:06.898066 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 17 19:14:06.898066 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 17 19:14:06.898066 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:14:06.898066 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:14:06.898066 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 621/1242] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:14:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:14:06.903123 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bbbad1-5b96-45ed-9bd5-cbd9a1435e26 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:14:06.998098 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bbbad1-5b96-45ed-9bd5-cbd9a1435e26 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:14:07.030505 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12850a8d-7a19-40bb-9435-af9838364d82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:14:07.031151 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12850a8d-7a19-40bb-9435-af9838364d82 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:14:07.031471 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12850a8d-7a19-40bb-9435-af9838364d82 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:14:07.031765 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12850a8d-7a19-40bb-9435-af9838364d82 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:14:07.032462 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12850a8d-7a19-40bb-9435-af9838364d82 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:14:07.032755 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12850a8d-7a19-40bb-9435-af9838364d82 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:14:07.034130 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12850a8d-7a19-40bb-9435-af9838364d82 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:14:07.045305 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 622/1243] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 17 19:14:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:14:07.050934 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4baf2f1-8967-45f0-80ce-96943fe5d6f9 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:14:07.144799 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4baf2f1-8967-45f0-80ce-96943fe5d6f9 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:14:07.177386 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf81995-6e50-459a-b685-4854d4e0c323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:14:07.178157 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf81995-6e50-459a-b685-4854d4e0c323 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:14:07.178433 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf81995-6e50-459a-b685-4854d4e0c323 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:14:07.178699 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf81995-6e50-459a-b685-4854d4e0c323 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:14:07.185781 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf81995-6e50-459a-b685-4854d4e0c323 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:14:07.187311 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf81995-6e50-459a-b685-4854d4e0c323 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:14:07.200706 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-ebf81995-6e50-459a-b685-4854d4e0c323 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 17 19:14:07.200706 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:14:07.200706 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:14:07.200706 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:14:07.200706 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:14:07.200706 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:14:07.200706 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:14:07.200706 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:14:07.200706 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:14:07.200706 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:14:07.200706 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:14:07.200706 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:14:07.200706 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 17 19:14:07.200706 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(project_id) May 17 19:14:07.200706 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 17 19:14:07.204528 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 17 19:14:07.204528 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:14:07.204528 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:14:07.204528 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:14:07.204528 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:14:07.204528 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:14:07.204528 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:14:07.204528 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:14:07.204528 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:14:07.204528 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:14:07.204528 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:14:07.204528 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:14:07.204528 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:14:07.204528 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:14:07.204528 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:14:07.204528 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:14:07.204528 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:14:07.205732 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 17 19:14:07.205732 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 17 19:14:07.205732 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 17 19:14:07.205732 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 17 19:14:07.205732 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 17 19:14:07.205732 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 17 19:14:07.205732 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 17 19:14:07.205732 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:14:07.205732 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 622/1244] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 17 19:14:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:14:07.207741 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12850a8d-7a19-40bb-9435-af9838364d82 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:14:07.307887 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12850a8d-7a19-40bb-9435-af9838364d82 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:14:07.339894 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae2c9ea-49c1-499c-aab4-70900fbd7da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:14:07.340535 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae2c9ea-49c1-499c-aab4-70900fbd7da1 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:14:07.340815 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae2c9ea-49c1-499c-aab4-70900fbd7da1 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:14:07.341089 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae2c9ea-49c1-499c-aab4-70900fbd7da1 None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:14:07.341860 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae2c9ea-49c1-499c-aab4-70900fbd7da1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:14:07.342165 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae2c9ea-49c1-499c-aab4-70900fbd7da1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:14:07.343616 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae2c9ea-49c1-499c-aab4-70900fbd7da1 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:14:07.355713 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 623/1245] 10.222.0.58 () {56 vars in 1148 bytes} [Tue May 17 19:14:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:14:07.361709 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf81995-6e50-459a-b685-4854d4e0c323 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:14:07.456335 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf81995-6e50-459a-b685-4854d4e0c323 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:14:07.488697 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3163160-de1b-4125-8707-50d78fe9873f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:14:07.489351 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3163160-de1b-4125-8707-50d78fe9873f None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:14:07.489631 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3163160-de1b-4125-8707-50d78fe9873f None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:14:07.489901 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3163160-de1b-4125-8707-50d78fe9873f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:14:07.490721 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3163160-de1b-4125-8707-50d78fe9873f None admin] RBAC: Adding query filter params (role.id=1754a95b778c4e20b53b20cb52ada92b, scope.project.id=1a9d3a0631284c9fb113266036993e2d, user.id=06538aab5f194af7b7c677d6b94aef2a) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:14:07.491027 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3163160-de1b-4125-8707-50d78fe9873f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:14:07.492470 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3163160-de1b-4125-8707-50d78fe9873f None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:14:07.498131 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 623/1246] 10.222.0.58 () {56 vars in 1374 bytes} [Tue May 17 19:14:07 2022] GET /identity/v3/role_assignments?user.id=06538aab5f194af7b7c677d6b94aef2a&scope.project.id=1a9d3a0631284c9fb113266036993e2d&role.id=1754a95b778c4e20b53b20cb52ada92b => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:14:09.726061 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74912443-a66f-4079-9c92-35fe994e607e None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:14:09.726601 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74912443-a66f-4079-9c92-35fe994e607e None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:14:09.726601 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74912443-a66f-4079-9c92-35fe994e607e None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:14:09.728047 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 624/1247] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:14:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:14:09.733895 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc9630a-ea8c-4986-88dd-b18123db739d None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:14:09.734281 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc9630a-ea8c-4986-88dd-b18123db739d None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:14:09.734448 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc9630a-ea8c-4986-88dd-b18123db739d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:14:09.750740 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-0bc9630a-ea8c-4986-88dd-b18123db739d None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:14:09.751953 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-0bc9630a-ea8c-4986-88dd-b18123db739d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:14:09.814963 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 624/1248] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:14:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 17 19:14:09.837338 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74912443-a66f-4079-9c92-35fe994e607e None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:14:09.934920 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74912443-a66f-4079-9c92-35fe994e607e None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:14:09.968397 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76848640-973b-49ce-851f-549effcd1cde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:14:09.970325 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76848640-973b-49ce-851f-549effcd1cde None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:14:09.970325 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76848640-973b-49ce-851f-549effcd1cde None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:14:09.970325 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76848640-973b-49ce-851f-549effcd1cde None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:14:09.988092 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76848640-973b-49ce-851f-549effcd1cde None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 17 19:14:09.988343 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-76848640-973b-49ce-851f-549effcd1cde None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 17 19:14:09.988343 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:14:09.988343 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:14:09.988343 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:14:09.988343 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:14:09.988343 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:14:09.988343 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:14:09.988343 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:14:09.988343 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:14:09.988343 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:14:09.988343 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:14:09.988343 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:14:09.988343 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 17 19:14:09.988343 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) May 17 19:14:09.988343 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 17 19:14:09.988343 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 17 19:14:09.990093 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 17 19:14:09.990093 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 17 19:14:09.990093 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 17 19:14:09.990093 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 17 19:14:09.990093 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:14:09.990093 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:14:09.990093 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:14:09.990093 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:14:09.990093 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:14:09.990093 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:14:09.990093 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:14:09.990093 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:14:09.990093 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:14:09.990093 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:14:09.990093 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:14:09.990093 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:14:09.991762 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:14:09.991762 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:14:09.991762 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:14:09.991762 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:14:09.991762 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 17 19:14:09.991762 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 17 19:14:09.991762 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 17 19:14:09.991762 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 17 19:14:09.991762 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 17 19:14:09.991762 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 17 19:14:09.991762 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 17 19:14:09.991762 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:14:09.991762 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 625/1249] 10.222.0.58 () {56 vars in 1096 bytes} [Tue May 17 19:14:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:14:09.996094 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc9630a-ea8c-4986-88dd-b18123db739d None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:14:10.091590 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc9630a-ea8c-4986-88dd-b18123db739d None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:14:10.124075 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1558cde5-4352-4f74-a63b-c8bdf5a97d27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:14:10.124722 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1558cde5-4352-4f74-a63b-c8bdf5a97d27 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:14:10.125000 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1558cde5-4352-4f74-a63b-c8bdf5a97d27 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:14:10.125268 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1558cde5-4352-4f74-a63b-c8bdf5a97d27 None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:14:10.125996 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1558cde5-4352-4f74-a63b-c8bdf5a97d27 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:14:10.126293 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1558cde5-4352-4f74-a63b-c8bdf5a97d27 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:14:10.127673 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1558cde5-4352-4f74-a63b-c8bdf5a97d27 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:14:10.135862 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 625/1250] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 17 19:14:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:14:10.142416 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76848640-973b-49ce-851f-549effcd1cde None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:14:10.238513 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76848640-973b-49ce-851f-549effcd1cde None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:14:10.270420 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37907f1d-fe65-40dc-9573-6da9425b0aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:14:10.271196 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37907f1d-fe65-40dc-9573-6da9425b0aa2 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:14:10.271473 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37907f1d-fe65-40dc-9573-6da9425b0aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:14:10.271774 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37907f1d-fe65-40dc-9573-6da9425b0aa2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:14:10.282410 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37907f1d-fe65-40dc-9573-6da9425b0aa2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:14:10.286288 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37907f1d-fe65-40dc-9573-6da9425b0aa2 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:14:10.300920 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-37907f1d-fe65-40dc-9573-6da9425b0aa2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:14:10.300920 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:14:10.300920 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:14:10.300920 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:14:10.300920 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:14:10.300920 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:14:10.300920 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:14:10.300920 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:14:10.300920 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:14:10.300920 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:14:10.300920 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:14:10.300920 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:14:10.300920 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 17 19:14:10.300920 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 17 19:14:10.300920 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 17 19:14:10.302029 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 17 19:14:10.302029 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:14:10.302029 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:14:10.302029 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:14:10.302029 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:14:10.302029 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:14:10.302029 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:14:10.302029 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:14:10.302029 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:14:10.302029 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:14:10.302029 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:14:10.302029 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:14:10.302029 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:14:10.302029 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:14:10.302029 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:14:10.302029 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:14:10.302029 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:14:10.307804 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 17 19:14:10.307804 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 17 19:14:10.307804 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:14:10.307804 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:14:10.307804 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 626/1251] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:14:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:14:10.308171 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1558cde5-4352-4f74-a63b-c8bdf5a97d27 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:14:10.402831 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1558cde5-4352-4f74-a63b-c8bdf5a97d27 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:14:10.434710 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082948f9-01e3-46c8-911d-46c36bc6e082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:14:10.435352 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082948f9-01e3-46c8-911d-46c36bc6e082 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:14:10.435721 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082948f9-01e3-46c8-911d-46c36bc6e082 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:14:10.435990 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082948f9-01e3-46c8-911d-46c36bc6e082 None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:14:10.436696 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082948f9-01e3-46c8-911d-46c36bc6e082 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:14:10.436992 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082948f9-01e3-46c8-911d-46c36bc6e082 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:14:10.438340 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082948f9-01e3-46c8-911d-46c36bc6e082 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:14:10.449490 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 626/1252] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 17 19:14:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:14:10.455043 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37907f1d-fe65-40dc-9573-6da9425b0aa2 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:14:10.551039 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37907f1d-fe65-40dc-9573-6da9425b0aa2 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:14:10.583714 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ee2380-0460-4d4c-979c-9e0de76fba34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:14:10.584498 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ee2380-0460-4d4c-979c-9e0de76fba34 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:14:10.584777 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ee2380-0460-4d4c-979c-9e0de76fba34 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:14:10.585049 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ee2380-0460-4d4c-979c-9e0de76fba34 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:14:10.592207 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ee2380-0460-4d4c-979c-9e0de76fba34 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:14:10.594045 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ee2380-0460-4d4c-979c-9e0de76fba34 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:14:10.607751 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-e8ee2380-0460-4d4c-979c-9e0de76fba34 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 17 19:14:10.607751 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:14:10.607751 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:14:10.607751 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:14:10.607751 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:14:10.607751 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:14:10.607751 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:14:10.607751 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:14:10.607751 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:14:10.607751 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:14:10.607751 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:14:10.607751 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:14:10.607751 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 17 19:14:10.607751 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_user(user_id) May 17 19:14:10.607751 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 17 19:14:10.607751 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 17 19:14:10.608960 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:14:10.608960 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:14:10.608960 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 17 19:14:10.608960 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 17 19:14:10.608960 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 17 19:14:10.608960 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 17 19:14:10.608960 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:14:10.608960 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:14:10.608960 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:14:10.608960 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:14:10.608960 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:14:10.608960 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:14:10.608960 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:14:10.608960 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:14:10.608960 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:14:10.608960 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:14:10.608960 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:14:10.608960 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:14:10.610074 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:14:10.610074 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:14:10.610074 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 17 19:14:10.610074 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 17 19:14:10.610074 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 17 19:14:10.610074 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 17 19:14:10.610074 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 17 19:14:10.610074 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 17 19:14:10.610074 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 17 19:14:10.610074 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:14:10.610074 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 627/1253] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:14:10 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:14:10.614545 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082948f9-01e3-46c8-911d-46c36bc6e082 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:14:10.709510 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082948f9-01e3-46c8-911d-46c36bc6e082 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:14:10.741938 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c9f2de-f7a8-41d6-9436-7f54c2d3f1e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:14:10.742591 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c9f2de-f7a8-41d6-9436-7f54c2d3f1e0 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:14:10.742865 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c9f2de-f7a8-41d6-9436-7f54c2d3f1e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:14:10.743129 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c9f2de-f7a8-41d6-9436-7f54c2d3f1e0 None admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:14:10.743973 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c9f2de-f7a8-41d6-9436-7f54c2d3f1e0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:14:10.744272 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c9f2de-f7a8-41d6-9436-7f54c2d3f1e0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:14:10.746278 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c9f2de-f7a8-41d6-9436-7f54c2d3f1e0 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:14:10.762495 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 627/1254] 10.222.0.58 () {56 vars in 1143 bytes} [Tue May 17 19:14:10 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:14:10.768201 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ee2380-0460-4d4c-979c-9e0de76fba34 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:14:10.863528 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ee2380-0460-4d4c-979c-9e0de76fba34 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:14:10.895300 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ad866c-1cca-42fc-81e8-82d549b670ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:14:10.895964 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ad866c-1cca-42fc-81e8-82d549b670ae None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:14:10.896244 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ad866c-1cca-42fc-81e8-82d549b670ae None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:14:10.896515 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ad866c-1cca-42fc-81e8-82d549b670ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:14:10.959420 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ad866c-1cca-42fc-81e8-82d549b670ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:14:10.960928 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ad866c-1cca-42fc-81e8-82d549b670ae None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:14:11.053036 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 628/1255] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 17 19:14:10 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 17 19:14:11.059447 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c9f2de-f7a8-41d6-9436-7f54c2d3f1e0 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:14:11.154972 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c9f2de-f7a8-41d6-9436-7f54c2d3f1e0 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:14:11.188156 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1140af-06bf-47b1-9305-510452dc7e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:14:11.188944 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1140af-06bf-47b1-9305-510452dc7e18 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:14:11.189228 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1140af-06bf-47b1-9305-510452dc7e18 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:14:11.189499 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1140af-06bf-47b1-9305-510452dc7e18 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:14:11.199794 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1140af-06bf-47b1-9305-510452dc7e18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:14:11.201374 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1140af-06bf-47b1-9305-510452dc7e18 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:14:11.218255 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-5b1140af-06bf-47b1-9305-510452dc7e18 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:14:11.218255 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:14:11.218255 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:14:11.218255 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:14:11.218255 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:14:11.218255 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:14:11.218255 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:14:11.218255 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:14:11.218255 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:14:11.218255 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:14:11.218255 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:14:11.218255 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:14:11.218255 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 17 19:14:11.218255 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 17 19:14:11.218255 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 17 19:14:11.219322 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 17 19:14:11.219322 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:14:11.219322 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:14:11.219322 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:14:11.219322 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:14:11.219322 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:14:11.219322 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:14:11.219322 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:14:11.219322 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:14:11.219322 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:14:11.219322 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:14:11.219322 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:14:11.219322 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:14:11.219322 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:14:11.219322 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:14:11.219322 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:14:11.219322 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:14:11.220899 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 17 19:14:11.220899 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 17 19:14:11.220899 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:14:11.220899 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:14:11.220899 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 628/1256] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:14:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:14:11.225353 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ad866c-1cca-42fc-81e8-82d549b670ae None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:14:11.319605 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ad866c-1cca-42fc-81e8-82d549b670ae None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:14:11.352178 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d9495c-cdf9-4c78-b814-962c714d0607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:14:11.352860 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d9495c-cdf9-4c78-b814-962c714d0607 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:14:11.353135 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d9495c-cdf9-4c78-b814-962c714d0607 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:14:11.353402 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d9495c-cdf9-4c78-b814-962c714d0607 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:14:11.354135 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d9495c-cdf9-4c78-b814-962c714d0607 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:14:11.354435 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d9495c-cdf9-4c78-b814-962c714d0607 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:14:11.355816 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d9495c-cdf9-4c78-b814-962c714d0607 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:14:11.366450 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 629/1257] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 17 19:14:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:14:11.372557 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1140af-06bf-47b1-9305-510452dc7e18 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:14:11.466384 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1140af-06bf-47b1-9305-510452dc7e18 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:14:11.502530 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfe521c-e94d-49a1-88ab-b40df630b827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:14:11.503298 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfe521c-e94d-49a1-88ab-b40df630b827 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:14:11.503603 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfe521c-e94d-49a1-88ab-b40df630b827 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:14:11.503876 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfe521c-e94d-49a1-88ab-b40df630b827 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:14:11.510313 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfe521c-e94d-49a1-88ab-b40df630b827 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:14:11.519224 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfe521c-e94d-49a1-88ab-b40df630b827 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:14:11.525265 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-8bfe521c-e94d-49a1-88ab-b40df630b827 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 17 19:14:11.525265 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:14:11.525265 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:14:11.525265 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:14:11.525265 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:14:11.525265 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:14:11.525265 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:14:11.525265 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:14:11.525265 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:14:11.525265 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:14:11.525265 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:14:11.525265 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:14:11.525265 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 17 19:14:11.525265 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(project_id) May 17 19:14:11.525265 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 17 19:14:11.526338 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 17 19:14:11.526338 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:14:11.526338 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:14:11.526338 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:14:11.526338 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:14:11.526338 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:14:11.526338 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:14:11.526338 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:14:11.526338 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:14:11.526338 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:14:11.526338 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:14:11.526338 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:14:11.526338 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:14:11.526338 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:14:11.526338 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:14:11.526338 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:14:11.526338 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:14:11.527390 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 17 19:14:11.527390 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 17 19:14:11.527390 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 17 19:14:11.527390 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 17 19:14:11.527390 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 17 19:14:11.527390 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 17 19:14:11.527390 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 17 19:14:11.527390 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:14:11.527390 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 629/1258] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 17 19:14:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:14:11.532262 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d9495c-cdf9-4c78-b814-962c714d0607 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:14:11.627914 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d9495c-cdf9-4c78-b814-962c714d0607 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:14:11.660401 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44731f7a-2d5e-4411-9940-c1a33861a2f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:14:11.662208 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44731f7a-2d5e-4411-9940-c1a33861a2f6 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:14:11.662484 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44731f7a-2d5e-4411-9940-c1a33861a2f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:14:11.662750 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44731f7a-2d5e-4411-9940-c1a33861a2f6 None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:14:11.663533 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44731f7a-2d5e-4411-9940-c1a33861a2f6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:14:11.670548 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44731f7a-2d5e-4411-9940-c1a33861a2f6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:14:11.670548 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44731f7a-2d5e-4411-9940-c1a33861a2f6 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:14:11.676717 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 630/1259] 10.222.0.58 () {56 vars in 1148 bytes} [Tue May 17 19:14:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:14:11.682914 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfe521c-e94d-49a1-88ab-b40df630b827 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:14:11.779913 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfe521c-e94d-49a1-88ab-b40df630b827 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:14:11.811656 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c97cc8-0597-4801-bec4-bca1b3fe76c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:14:11.812295 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c97cc8-0597-4801-bec4-bca1b3fe76c0 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:14:11.812610 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c97cc8-0597-4801-bec4-bca1b3fe76c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:14:11.812841 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c97cc8-0597-4801-bec4-bca1b3fe76c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:14:11.813655 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c97cc8-0597-4801-bec4-bca1b3fe76c0 None admin] RBAC: Adding query filter params (role.id=60af5088793e4379b88821a86552a344, scope.project.id=1a9d3a0631284c9fb113266036993e2d, user.id=06538aab5f194af7b7c677d6b94aef2a) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:14:11.813955 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c97cc8-0597-4801-bec4-bca1b3fe76c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:14:11.815367 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c97cc8-0597-4801-bec4-bca1b3fe76c0 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:14:11.820614 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 630/1260] 10.222.0.58 () {56 vars in 1374 bytes} [Tue May 17 19:14:11 2022] GET /identity/v3/role_assignments?user.id=06538aab5f194af7b7c677d6b94aef2a&scope.project.id=1a9d3a0631284c9fb113266036993e2d&role.id=60af5088793e4379b88821a86552a344 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:14:13.969957 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fe5965-b154-4aaa-bfde-d218b10c8c2d None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:14:13.970629 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fe5965-b154-4aaa-bfde-d218b10c8c2d None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:14:13.970750 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fe5965-b154-4aaa-bfde-d218b10c8c2d None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:14:13.972262 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 631/1261] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:14:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:14:13.977455 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796badde-d7f7-4338-b976-68509b2ae2d4 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:14:13.977891 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796badde-d7f7-4338-b976-68509b2ae2d4 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:14:13.978011 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796badde-d7f7-4338-b976-68509b2ae2d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:14:13.994243 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-796badde-d7f7-4338-b976-68509b2ae2d4 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:14:13.995411 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-796badde-d7f7-4338-b976-68509b2ae2d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:14:14.059353 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 631/1262] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:14:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 17 19:14:14.082334 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fe5965-b154-4aaa-bfde-d218b10c8c2d None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:14:14.180896 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fe5965-b154-4aaa-bfde-d218b10c8c2d None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:14:14.213532 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25ffa52-f0fc-4e54-ad7c-faadad0195bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:14:14.214243 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25ffa52-f0fc-4e54-ad7c-faadad0195bf None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:14:14.214514 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25ffa52-f0fc-4e54-ad7c-faadad0195bf None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:14:14.214779 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25ffa52-f0fc-4e54-ad7c-faadad0195bf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:14:14.224732 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25ffa52-f0fc-4e54-ad7c-faadad0195bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 17 19:14:14.232495 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-c25ffa52-f0fc-4e54-ad7c-faadad0195bf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 17 19:14:14.232495 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:14:14.232495 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:14:14.232495 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:14:14.232495 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:14:14.232495 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:14:14.232495 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:14:14.232495 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:14:14.232495 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:14:14.232495 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:14:14.232495 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:14:14.232495 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:14:14.232495 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 17 19:14:14.232495 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) May 17 19:14:14.232495 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 17 19:14:14.232495 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 17 19:14:14.233619 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 17 19:14:14.233619 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 17 19:14:14.233619 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 17 19:14:14.233619 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 17 19:14:14.233619 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:14:14.233619 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:14:14.233619 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:14:14.233619 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:14:14.233619 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:14:14.233619 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:14:14.233619 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:14:14.233619 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:14:14.233619 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:14:14.233619 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:14:14.233619 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:14:14.233619 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:14:14.234867 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:14:14.234867 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:14:14.234867 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:14:14.234867 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:14:14.234867 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 17 19:14:14.234867 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 17 19:14:14.234867 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 17 19:14:14.234867 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 17 19:14:14.234867 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 17 19:14:14.234867 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 17 19:14:14.234867 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 17 19:14:14.234867 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:14:14.234867 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 632/1263] 10.222.0.58 () {56 vars in 1096 bytes} [Tue May 17 19:14:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:14:14.239634 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796badde-d7f7-4338-b976-68509b2ae2d4 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:14:14.332641 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796badde-d7f7-4338-b976-68509b2ae2d4 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:14:14.364304 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a7b5c5-83f1-42b3-9513-131d278f0ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:14:14.364944 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a7b5c5-83f1-42b3-9513-131d278f0ae5 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:14:14.365219 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a7b5c5-83f1-42b3-9513-131d278f0ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:14:14.365486 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a7b5c5-83f1-42b3-9513-131d278f0ae5 None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:14:14.366217 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a7b5c5-83f1-42b3-9513-131d278f0ae5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:14:14.366518 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a7b5c5-83f1-42b3-9513-131d278f0ae5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:14:14.367903 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a7b5c5-83f1-42b3-9513-131d278f0ae5 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:14:14.376055 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 632/1264] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 17 19:14:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:14:14.381815 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25ffa52-f0fc-4e54-ad7c-faadad0195bf None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:14:14.476004 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25ffa52-f0fc-4e54-ad7c-faadad0195bf None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:14:14.508348 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54d35cc-0f28-44d1-b13b-0177d22334c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:14:14.509116 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54d35cc-0f28-44d1-b13b-0177d22334c4 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:14:14.509388 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54d35cc-0f28-44d1-b13b-0177d22334c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:14:14.509652 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54d35cc-0f28-44d1-b13b-0177d22334c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:14:14.519592 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54d35cc-0f28-44d1-b13b-0177d22334c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:14:14.521014 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54d35cc-0f28-44d1-b13b-0177d22334c4 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:14:14.537124 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-b54d35cc-0f28-44d1-b13b-0177d22334c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:14:14.537124 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:14:14.537124 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:14:14.537124 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:14:14.537124 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:14:14.537124 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:14:14.537124 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:14:14.537124 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:14:14.537124 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:14:14.537124 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:14:14.537124 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:14:14.537124 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:14:14.537124 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 17 19:14:14.537124 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 17 19:14:14.537124 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 17 19:14:14.538018 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 17 19:14:14.538018 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:14:14.538018 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:14:14.538018 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:14:14.538018 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:14:14.538018 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:14:14.538018 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:14:14.538018 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:14:14.538018 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:14:14.538018 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:14:14.538018 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:14:14.538018 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:14:14.538018 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:14:14.538018 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:14:14.538018 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:14:14.538018 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:14:14.538018 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:14:14.538924 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 17 19:14:14.538924 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 17 19:14:14.538924 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:14:14.538924 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:14:14.538924 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 633/1265] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:14:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:14:14.545003 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a7b5c5-83f1-42b3-9513-131d278f0ae5 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:14:14.641622 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a7b5c5-83f1-42b3-9513-131d278f0ae5 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:14:14.673569 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8238c539-1c55-4ed4-9284-b56595f46b79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:14:14.674211 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8238c539-1c55-4ed4-9284-b56595f46b79 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:14:14.674488 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8238c539-1c55-4ed4-9284-b56595f46b79 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:14:14.674760 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8238c539-1c55-4ed4-9284-b56595f46b79 None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:14:14.675467 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8238c539-1c55-4ed4-9284-b56595f46b79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:14:14.675800 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8238c539-1c55-4ed4-9284-b56595f46b79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:14:14.677152 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8238c539-1c55-4ed4-9284-b56595f46b79 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:14:14.688959 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 633/1266] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 17 19:14:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:14:14.694590 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54d35cc-0f28-44d1-b13b-0177d22334c4 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:14:14.789093 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54d35cc-0f28-44d1-b13b-0177d22334c4 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:14:14.821277 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c328a1da-cce1-442c-b8d5-cd4d0dc9ecf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:14:14.822057 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c328a1da-cce1-442c-b8d5-cd4d0dc9ecf8 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:14:14.822334 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c328a1da-cce1-442c-b8d5-cd4d0dc9ecf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:14:14.822602 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c328a1da-cce1-442c-b8d5-cd4d0dc9ecf8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:14:14.829733 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c328a1da-cce1-442c-b8d5-cd4d0dc9ecf8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:14:14.831161 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c328a1da-cce1-442c-b8d5-cd4d0dc9ecf8 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:14:14.846087 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-c328a1da-cce1-442c-b8d5-cd4d0dc9ecf8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 17 19:14:14.846087 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:14:14.846087 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:14:14.846087 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:14:14.846087 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:14:14.846087 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:14:14.846087 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:14:14.846087 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:14:14.846087 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:14:14.846087 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:14:14.846087 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:14:14.846087 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:14:14.846087 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 17 19:14:14.846087 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_user(user_id) May 17 19:14:14.846087 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 17 19:14:14.846087 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 17 19:14:14.847232 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:14:14.847232 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:14:14.847232 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 17 19:14:14.847232 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 17 19:14:14.847232 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 17 19:14:14.847232 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 17 19:14:14.847232 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:14:14.847232 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:14:14.847232 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:14:14.847232 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:14:14.847232 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:14:14.847232 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:14:14.847232 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:14:14.847232 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:14:14.847232 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:14:14.847232 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:14:14.847232 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:14:14.847232 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:14:14.848276 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:14:14.848276 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:14:14.848276 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 17 19:14:14.848276 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 17 19:14:14.848276 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 17 19:14:14.848276 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 17 19:14:14.848276 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 17 19:14:14.848276 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 17 19:14:14.848276 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 17 19:14:14.848276 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:14:14.848276 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 634/1267] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:14:14 2022] GET /identity/v3/users/placement => generated 87 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:14:14.852960 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8238c539-1c55-4ed4-9284-b56595f46b79 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:14:14.945946 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8238c539-1c55-4ed4-9284-b56595f46b79 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:14:14.978370 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dffb3f7-4c32-4efc-aa26-e871b5f3eefd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:14:14.979816 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dffb3f7-4c32-4efc-aa26-e871b5f3eefd None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:14:14.979922 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dffb3f7-4c32-4efc-aa26-e871b5f3eefd None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:14:14.980201 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dffb3f7-4c32-4efc-aa26-e871b5f3eefd None admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:14:14.981027 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dffb3f7-4c32-4efc-aa26-e871b5f3eefd None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:14:14.981327 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dffb3f7-4c32-4efc-aa26-e871b5f3eefd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:14:14.982746 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dffb3f7-4c32-4efc-aa26-e871b5f3eefd None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:14:14.999641 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 634/1268] 10.222.0.58 () {56 vars in 1143 bytes} [Tue May 17 19:14:14 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:14:15.005313 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c328a1da-cce1-442c-b8d5-cd4d0dc9ecf8 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:14:15.109073 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c328a1da-cce1-442c-b8d5-cd4d0dc9ecf8 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:14:15.135354 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1f71f9-4d8e-4a97-b61e-cb253b01d93f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:14:15.136178 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1f71f9-4d8e-4a97-b61e-cb253b01d93f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:14:15.141036 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1f71f9-4d8e-4a97-b61e-cb253b01d93f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:14:15.141036 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1f71f9-4d8e-4a97-b61e-cb253b01d93f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:14:15.146717 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1f71f9-4d8e-4a97-b61e-cb253b01d93f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:14:15.148194 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1f71f9-4d8e-4a97-b61e-cb253b01d93f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:14:15.164883 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-2e1f71f9-4d8e-4a97-b61e-cb253b01d93f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:14:15.164883 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:14:15.164883 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:14:15.164883 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:14:15.164883 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:14:15.164883 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:14:15.164883 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:14:15.164883 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:14:15.164883 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:14:15.164883 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:14:15.164883 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:14:15.164883 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:14:15.164883 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 17 19:14:15.164883 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 17 19:14:15.164883 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 17 19:14:15.165943 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 17 19:14:15.165943 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:14:15.165943 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:14:15.165943 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:14:15.165943 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:14:15.165943 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:14:15.165943 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:14:15.165943 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:14:15.165943 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:14:15.165943 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:14:15.165943 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:14:15.165943 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:14:15.165943 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:14:15.165943 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:14:15.165943 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:14:15.165943 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:14:15.165943 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:14:15.167809 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 17 19:14:15.167809 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 17 19:14:15.167809 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:14:15.167809 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:14:15.167809 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 635/1269] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:14:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:14:15.172271 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dffb3f7-4c32-4efc-aa26-e871b5f3eefd None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:14:15.266181 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dffb3f7-4c32-4efc-aa26-e871b5f3eefd None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:14:15.298202 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d42b8a-b97c-4ca4-a50c-c1f04de227bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:14:15.298840 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d42b8a-b97c-4ca4-a50c-c1f04de227bc None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:14:15.299113 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d42b8a-b97c-4ca4-a50c-c1f04de227bc None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:14:15.299398 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d42b8a-b97c-4ca4-a50c-c1f04de227bc None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:14:15.300110 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d42b8a-b97c-4ca4-a50c-c1f04de227bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:14:15.300405 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d42b8a-b97c-4ca4-a50c-c1f04de227bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:14:15.301819 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d42b8a-b97c-4ca4-a50c-c1f04de227bc None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:14:15.312463 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 635/1270] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 17 19:14:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:14:15.318105 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1f71f9-4d8e-4a97-b61e-cb253b01d93f None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:14:15.413887 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1f71f9-4d8e-4a97-b61e-cb253b01d93f None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:14:15.445391 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916fd8a8-ea96-4858-bcad-4fbdd3df2251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:14:15.446163 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916fd8a8-ea96-4858-bcad-4fbdd3df2251 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:14:15.446443 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916fd8a8-ea96-4858-bcad-4fbdd3df2251 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:14:15.446715 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916fd8a8-ea96-4858-bcad-4fbdd3df2251 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:14:15.453717 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916fd8a8-ea96-4858-bcad-4fbdd3df2251 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:14:15.455150 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916fd8a8-ea96-4858-bcad-4fbdd3df2251 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:14:15.469739 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-916fd8a8-ea96-4858-bcad-4fbdd3df2251 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 17 19:14:15.469739 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:14:15.469739 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:14:15.469739 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:14:15.469739 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:14:15.469739 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:14:15.469739 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:14:15.469739 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:14:15.469739 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:14:15.469739 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:14:15.469739 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:14:15.469739 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:14:15.469739 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 17 19:14:15.469739 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(project_id) May 17 19:14:15.469739 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 17 19:14:15.471159 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 17 19:14:15.471159 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:14:15.471159 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:14:15.471159 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:14:15.471159 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:14:15.471159 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:14:15.471159 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:14:15.471159 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:14:15.471159 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:14:15.471159 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:14:15.471159 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:14:15.471159 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:14:15.471159 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:14:15.471159 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:14:15.471159 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:14:15.471159 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:14:15.471159 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:14:15.472440 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 17 19:14:15.472440 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 17 19:14:15.472440 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 17 19:14:15.472440 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 17 19:14:15.472440 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 17 19:14:15.472440 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 17 19:14:15.472440 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 17 19:14:15.472440 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:14:15.472440 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 636/1271] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 17 19:14:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:14:15.477056 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d42b8a-b97c-4ca4-a50c-c1f04de227bc None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:14:15.570824 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d42b8a-b97c-4ca4-a50c-c1f04de227bc None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:14:15.603043 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20ebcaa-a587-4815-a1ef-ba2bd7ef49df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:14:15.603754 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20ebcaa-a587-4815-a1ef-ba2bd7ef49df None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:14:15.604028 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20ebcaa-a587-4815-a1ef-ba2bd7ef49df None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:14:15.604296 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20ebcaa-a587-4815-a1ef-ba2bd7ef49df None admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:14:15.605066 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20ebcaa-a587-4815-a1ef-ba2bd7ef49df None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:14:15.605368 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20ebcaa-a587-4815-a1ef-ba2bd7ef49df None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:14:15.606774 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20ebcaa-a587-4815-a1ef-ba2bd7ef49df None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:14:15.618359 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 636/1272] 10.222.0.58 () {56 vars in 1148 bytes} [Tue May 17 19:14:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:14:15.624773 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916fd8a8-ea96-4858-bcad-4fbdd3df2251 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:14:15.723017 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916fd8a8-ea96-4858-bcad-4fbdd3df2251 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:14:15.755893 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8980161f-832c-494e-8c5a-26904b4362b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:14:15.756764 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8980161f-832c-494e-8c5a-26904b4362b4 None admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:14:15.757035 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8980161f-832c-494e-8c5a-26904b4362b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:14:15.757301 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8980161f-832c-494e-8c5a-26904b4362b4 None admin] PATH_INFO: `/v3/projects/1a9d3a0631284c9fb113266036993e2d/users/06538aab5f194af7b7c677d6b94aef2a/roles/60af5088793e4379b88821a86552a344` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:14:15.759207 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8980161f-832c-494e-8c5a-26904b4362b4 None admin] RBAC: Authorizing `identity:create_grant(project_id=1a9d3a0631284c9fb113266036993e2d, user_id=06538aab5f194af7b7c677d6b94aef2a, role_id=60af5088793e4379b88821a86552a344)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:14:15.761269 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8980161f-832c-494e-8c5a-26904b4362b4 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:14:15.859664 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 637/1273] 10.222.0.58 () {58 vars in 1439 bytes} [Tue May 17 19:14:15 2022] PUT /identity/v3/projects/1a9d3a0631284c9fb113266036993e2d/users/06538aab5f194af7b7c677d6b94aef2a/roles/60af5088793e4379b88821a86552a344 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:14:18.041273 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4171a52b-004b-4b94-8f53-a306e5f6a471 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:14:18.042394 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4171a52b-004b-4b94-8f53-a306e5f6a471 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:14:18.042496 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4171a52b-004b-4b94-8f53-a306e5f6a471 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:14:18.043951 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 637/1274] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:14:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:14:18.049331 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727a6e23-9827-46b6-9926-d854ca9780fd None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:14:18.049776 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727a6e23-9827-46b6-9926-d854ca9780fd None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:14:18.049998 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727a6e23-9827-46b6-9926-d854ca9780fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:14:18.066646 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-727a6e23-9827-46b6-9926-d854ca9780fd None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:14:18.067871 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-727a6e23-9827-46b6-9926-d854ca9780fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:14:18.131386 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 638/1275] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:14:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 17 19:14:18.154667 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4171a52b-004b-4b94-8f53-a306e5f6a471 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:14:18.255775 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4171a52b-004b-4b94-8f53-a306e5f6a471 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:14:18.286801 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f768b1-792e-44cf-bc25-9e4cb165551a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:14:18.287598 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f768b1-792e-44cf-bc25-9e4cb165551a None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:14:18.287896 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f768b1-792e-44cf-bc25-9e4cb165551a None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:14:18.288164 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f768b1-792e-44cf-bc25-9e4cb165551a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:14:18.298434 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f768b1-792e-44cf-bc25-9e4cb165551a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 17 19:14:18.306080 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-17f768b1-792e-44cf-bc25-9e4cb165551a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 17 19:14:18.306080 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:14:18.306080 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:14:18.306080 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:14:18.306080 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:14:18.306080 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:14:18.306080 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:14:18.306080 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:14:18.306080 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:14:18.306080 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:14:18.306080 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:14:18.306080 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:14:18.306080 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 17 19:14:18.306080 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) May 17 19:14:18.306080 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 17 19:14:18.306080 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 17 19:14:18.307817 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 17 19:14:18.307817 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 17 19:14:18.307817 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 17 19:14:18.307817 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 17 19:14:18.307817 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:14:18.307817 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:14:18.307817 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:14:18.307817 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:14:18.307817 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:14:18.307817 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:14:18.307817 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:14:18.307817 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:14:18.307817 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:14:18.307817 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:14:18.307817 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:14:18.307817 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:14:18.308879 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:14:18.308879 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:14:18.308879 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:14:18.308879 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:14:18.308879 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 17 19:14:18.308879 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 17 19:14:18.308879 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 17 19:14:18.308879 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 17 19:14:18.308879 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 17 19:14:18.308879 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 17 19:14:18.308879 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 17 19:14:18.308879 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:14:18.308879 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 638/1276] 10.222.0.58 () {56 vars in 1096 bytes} [Tue May 17 19:14:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:14:18.313752 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727a6e23-9827-46b6-9926-d854ca9780fd None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:14:18.408106 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727a6e23-9827-46b6-9926-d854ca9780fd None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:14:18.441509 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57115ae5-2024-4036-9178-6baaab6a50dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:14:18.442169 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57115ae5-2024-4036-9178-6baaab6a50dc None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:14:18.442442 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57115ae5-2024-4036-9178-6baaab6a50dc None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:14:18.442710 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57115ae5-2024-4036-9178-6baaab6a50dc None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:14:18.443442 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57115ae5-2024-4036-9178-6baaab6a50dc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:14:18.443768 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57115ae5-2024-4036-9178-6baaab6a50dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:14:18.445124 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57115ae5-2024-4036-9178-6baaab6a50dc None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:14:18.453421 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 639/1277] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 17 19:14:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:14:18.459027 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f768b1-792e-44cf-bc25-9e4cb165551a None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:14:18.554357 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f768b1-792e-44cf-bc25-9e4cb165551a None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:14:18.585551 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b77ec1-fbfb-4f97-a744-a2f346e98d18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:14:18.586332 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b77ec1-fbfb-4f97-a744-a2f346e98d18 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:14:18.586608 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b77ec1-fbfb-4f97-a744-a2f346e98d18 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:14:18.586881 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b77ec1-fbfb-4f97-a744-a2f346e98d18 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:14:18.597827 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b77ec1-fbfb-4f97-a744-a2f346e98d18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:14:18.599250 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b77ec1-fbfb-4f97-a744-a2f346e98d18 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:14:18.616112 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-c4b77ec1-fbfb-4f97-a744-a2f346e98d18 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:14:18.616112 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:14:18.616112 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:14:18.616112 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:14:18.616112 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:14:18.616112 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:14:18.616112 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:14:18.616112 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:14:18.616112 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:14:18.616112 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:14:18.616112 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:14:18.616112 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:14:18.616112 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 17 19:14:18.616112 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 17 19:14:18.616112 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 17 19:14:18.617204 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 17 19:14:18.617204 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:14:18.617204 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:14:18.617204 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:14:18.617204 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:14:18.617204 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:14:18.617204 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:14:18.617204 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:14:18.617204 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:14:18.617204 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:14:18.617204 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:14:18.617204 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:14:18.617204 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:14:18.617204 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:14:18.617204 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:14:18.617204 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:14:18.617204 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:14:18.618172 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 17 19:14:18.618172 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 17 19:14:18.618172 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:14:18.618172 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:14:18.618172 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 639/1278] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:14:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:14:18.622941 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57115ae5-2024-4036-9178-6baaab6a50dc None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:14:18.717612 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57115ae5-2024-4036-9178-6baaab6a50dc None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:14:18.750618 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac75165b-0443-479d-a51e-20a058731a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:14:18.751266 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac75165b-0443-479d-a51e-20a058731a5e None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:14:18.751572 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac75165b-0443-479d-a51e-20a058731a5e None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:14:18.751841 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac75165b-0443-479d-a51e-20a058731a5e None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:14:18.752542 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac75165b-0443-479d-a51e-20a058731a5e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:14:18.752891 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac75165b-0443-479d-a51e-20a058731a5e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:14:18.754193 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac75165b-0443-479d-a51e-20a058731a5e None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:14:18.765435 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 640/1279] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 17 19:14:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:14:18.771023 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b77ec1-fbfb-4f97-a744-a2f346e98d18 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:14:18.869145 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b77ec1-fbfb-4f97-a744-a2f346e98d18 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:14:18.901496 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3760791-f003-4260-afc3-3cbd630eefbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:14:18.902280 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3760791-f003-4260-afc3-3cbd630eefbd None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:14:18.902556 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3760791-f003-4260-afc3-3cbd630eefbd None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:14:18.902828 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3760791-f003-4260-afc3-3cbd630eefbd None admin] PATH_INFO: `/v3/users/placement` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:14:18.910132 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3760791-f003-4260-afc3-3cbd630eefbd None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:14:18.911585 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3760791-f003-4260-afc3-3cbd630eefbd None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:14:18.925917 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-e3760791-f003-4260-afc3-3cbd630eefbd None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 17 19:14:18.925917 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:14:18.925917 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:14:18.925917 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:14:18.925917 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:14:18.925917 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:14:18.925917 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:14:18.925917 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:14:18.925917 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:14:18.925917 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:14:18.925917 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:14:18.925917 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:14:18.925917 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 17 19:14:18.925917 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_user(user_id) May 17 19:14:18.925917 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 17 19:14:18.925917 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 17 19:14:18.927089 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:14:18.927089 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:14:18.927089 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 17 19:14:18.927089 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 17 19:14:18.927089 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 17 19:14:18.927089 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 17 19:14:18.927089 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:14:18.927089 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:14:18.927089 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:14:18.927089 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:14:18.927089 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:14:18.927089 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:14:18.927089 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:14:18.927089 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:14:18.927089 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:14:18.927089 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:14:18.927089 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:14:18.927089 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:14:18.928251 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:14:18.928251 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:14:18.928251 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 17 19:14:18.928251 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 17 19:14:18.928251 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 17 19:14:18.928251 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 17 19:14:18.928251 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 17 19:14:18.928251 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 17 19:14:18.928251 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 17 19:14:18.928251 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:14:18.928251 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 640/1280] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:14:18 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:14:18.933725 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac75165b-0443-479d-a51e-20a058731a5e None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:14:19.030511 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac75165b-0443-479d-a51e-20a058731a5e None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:14:19.062563 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ceba28-476d-4748-8b3d-77b3db9ddc54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:14:19.063216 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ceba28-476d-4748-8b3d-77b3db9ddc54 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:14:19.063517 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ceba28-476d-4748-8b3d-77b3db9ddc54 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:14:19.063793 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ceba28-476d-4748-8b3d-77b3db9ddc54 None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:14:19.064615 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ceba28-476d-4748-8b3d-77b3db9ddc54 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:14:19.064918 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ceba28-476d-4748-8b3d-77b3db9ddc54 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:14:19.066343 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ceba28-476d-4748-8b3d-77b3db9ddc54 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:14:19.083292 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 641/1281] 10.222.0.58 () {56 vars in 1143 bytes} [Tue May 17 19:14:18 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:14:19.089759 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3760791-f003-4260-afc3-3cbd630eefbd None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:14:19.184287 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3760791-f003-4260-afc3-3cbd630eefbd None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:14:19.216351 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9332d937-5f4a-4b70-ad5a-4391982b7236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:14:19.216983 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9332d937-5f4a-4b70-ad5a-4391982b7236 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:14:19.217264 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9332d937-5f4a-4b70-ad5a-4391982b7236 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:14:19.217536 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9332d937-5f4a-4b70-ad5a-4391982b7236 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:14:19.280905 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9332d937-5f4a-4b70-ad5a-4391982b7236 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:14:19.282341 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9332d937-5f4a-4b70-ad5a-4391982b7236 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:14:19.374157 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 641/1282] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 17 19:14:19 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 17 19:14:19.380074 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ceba28-476d-4748-8b3d-77b3db9ddc54 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:14:19.475058 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ceba28-476d-4748-8b3d-77b3db9ddc54 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:14:19.507112 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b714e032-6fc0-45e5-a584-a401c1e2bfa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:14:19.507919 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b714e032-6fc0-45e5-a584-a401c1e2bfa1 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:14:19.508199 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b714e032-6fc0-45e5-a584-a401c1e2bfa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:14:19.508471 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b714e032-6fc0-45e5-a584-a401c1e2bfa1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:14:19.518734 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b714e032-6fc0-45e5-a584-a401c1e2bfa1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:14:19.520215 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b714e032-6fc0-45e5-a584-a401c1e2bfa1 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:14:19.537914 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-b714e032-6fc0-45e5-a584-a401c1e2bfa1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:14:19.537914 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:14:19.537914 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:14:19.537914 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:14:19.537914 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:14:19.537914 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:14:19.537914 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:14:19.537914 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:14:19.537914 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:14:19.537914 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:14:19.537914 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:14:19.537914 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:14:19.537914 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 17 19:14:19.537914 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 17 19:14:19.537914 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 17 19:14:19.539089 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 17 19:14:19.539089 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:14:19.539089 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:14:19.539089 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:14:19.539089 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:14:19.539089 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:14:19.539089 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:14:19.539089 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:14:19.539089 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:14:19.539089 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:14:19.539089 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:14:19.539089 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:14:19.539089 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:14:19.539089 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:14:19.539089 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:14:19.539089 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:14:19.539089 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:14:19.540234 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 17 19:14:19.540234 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 17 19:14:19.540234 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 17 19:14:19.540234 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:14:19.540234 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 642/1283] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:14:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:14:19.545269 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9332d937-5f4a-4b70-ad5a-4391982b7236 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:14:19.639968 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9332d937-5f4a-4b70-ad5a-4391982b7236 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:14:19.673071 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282b7600-7f6e-433e-9880-fdd7b2279233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:14:19.673723 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282b7600-7f6e-433e-9880-fdd7b2279233 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:14:19.674034 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282b7600-7f6e-433e-9880-fdd7b2279233 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:14:19.674308 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282b7600-7f6e-433e-9880-fdd7b2279233 None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:14:19.675020 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282b7600-7f6e-433e-9880-fdd7b2279233 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:14:19.675326 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282b7600-7f6e-433e-9880-fdd7b2279233 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:14:19.676726 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282b7600-7f6e-433e-9880-fdd7b2279233 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:14:19.688431 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 642/1284] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 17 19:14:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:14:19.694065 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b714e032-6fc0-45e5-a584-a401c1e2bfa1 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:14:19.789335 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b714e032-6fc0-45e5-a584-a401c1e2bfa1 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:14:19.824529 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26908e4-1d2e-4da3-b6b5-67bdf465119b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:14:19.825307 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26908e4-1d2e-4da3-b6b5-67bdf465119b None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:14:19.825580 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26908e4-1d2e-4da3-b6b5-67bdf465119b None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:14:19.825851 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26908e4-1d2e-4da3-b6b5-67bdf465119b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:14:19.832255 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26908e4-1d2e-4da3-b6b5-67bdf465119b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:14:19.833675 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26908e4-1d2e-4da3-b6b5-67bdf465119b None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:14:19.849946 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-a26908e4-1d2e-4da3-b6b5-67bdf465119b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 17 19:14:19.849946 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:14:19.849946 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:14:19.849946 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:14:19.849946 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:14:19.849946 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:14:19.849946 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:14:19.849946 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:14:19.849946 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:14:19.849946 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:14:19.849946 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:14:19.849946 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:14:19.849946 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 17 19:14:19.849946 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(project_id) May 17 19:14:19.849946 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 17 19:14:19.851070 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 17 19:14:19.851070 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:14:19.851070 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:14:19.851070 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:14:19.851070 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:14:19.851070 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:14:19.851070 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:14:19.851070 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:14:19.851070 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( May 17 19:14:19.851070 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:14:19.851070 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() May 17 19:14:19.851070 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:14:19.851070 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:14:19.851070 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:14:19.851070 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() May 17 19:14:19.851070 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:14:19.851070 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( May 17 19:14:19.867842 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 17 19:14:19.867842 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 17 19:14:19.867842 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 17 19:14:19.867842 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 17 19:14:19.867842 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 17 19:14:19.867842 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 17 19:14:19.867842 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 17 19:14:19.867842 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application May 17 19:14:19.867842 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 643/1285] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 17 19:14:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:14:19.868470 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282b7600-7f6e-433e-9880-fdd7b2279233 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:14:19.957125 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282b7600-7f6e-433e-9880-fdd7b2279233 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:14:19.989646 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77962b2-0333-45f3-8c62-7e16ad4401bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:14:19.990294 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77962b2-0333-45f3-8c62-7e16ad4401bc None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:14:19.990570 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77962b2-0333-45f3-8c62-7e16ad4401bc None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:14:19.990837 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77962b2-0333-45f3-8c62-7e16ad4401bc None admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:14:19.991638 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77962b2-0333-45f3-8c62-7e16ad4401bc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:14:19.991940 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77962b2-0333-45f3-8c62-7e16ad4401bc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:14:19.993504 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77962b2-0333-45f3-8c62-7e16ad4401bc None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:14:20.005768 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 643/1286] 10.222.0.58 () {56 vars in 1148 bytes} [Tue May 17 19:14:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:14:20.011718 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26908e4-1d2e-4da3-b6b5-67bdf465119b None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:14:20.108638 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26908e4-1d2e-4da3-b6b5-67bdf465119b None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:14:20.140455 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aacffa8-42d5-40f1-800a-bd782ef8d487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:14:20.141082 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aacffa8-42d5-40f1-800a-bd782ef8d487 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:14:20.141387 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aacffa8-42d5-40f1-800a-bd782ef8d487 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:14:20.141655 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aacffa8-42d5-40f1-800a-bd782ef8d487 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:14:20.142465 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aacffa8-42d5-40f1-800a-bd782ef8d487 None admin] RBAC: Adding query filter params (role.id=60af5088793e4379b88821a86552a344, scope.project.id=1a9d3a0631284c9fb113266036993e2d, user.id=06538aab5f194af7b7c677d6b94aef2a) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:14:20.142845 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aacffa8-42d5-40f1-800a-bd782ef8d487 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:14:20.144300 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aacffa8-42d5-40f1-800a-bd782ef8d487 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:14:20.149893 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 644/1287] 10.222.0.58 () {56 vars in 1374 bytes} [Tue May 17 19:14:20 2022] GET /identity/v3/role_assignments?user.id=06538aab5f194af7b7c677d6b94aef2a&scope.project.id=1a9d3a0631284c9fb113266036993e2d&role.id=60af5088793e4379b88821a86552a344 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:14:22.361104 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b767ad19-517b-4fa3-8818-68d08862de96 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:14:22.361717 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b767ad19-517b-4fa3-8818-68d08862de96 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:14:22.361717 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b767ad19-517b-4fa3-8818-68d08862de96 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:14:22.362955 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 644/1288] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:14:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:14:22.368233 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee47bb6-aff2-448e-bcb1-c56d338ce693 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:14:22.368656 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee47bb6-aff2-448e-bcb1-c56d338ce693 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:14:22.368767 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee47bb6-aff2-448e-bcb1-c56d338ce693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:14:22.386192 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-6ee47bb6-aff2-448e-bcb1-c56d338ce693 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:14:22.387391 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-6ee47bb6-aff2-448e-bcb1-c56d338ce693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:14:22.450322 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 645/1289] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:14:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 17 19:14:22.472571 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b767ad19-517b-4fa3-8818-68d08862de96 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:14:22.569646 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b767ad19-517b-4fa3-8818-68d08862de96 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:14:22.602441 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6edcbb-1385-4440-8189-a3f804512956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:14:22.603214 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6edcbb-1385-4440-8189-a3f804512956 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:14:22.603519 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6edcbb-1385-4440-8189-a3f804512956 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:14:22.603790 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6edcbb-1385-4440-8189-a3f804512956 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:14:22.605022 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6edcbb-1385-4440-8189-a3f804512956 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:14:22.606393 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6edcbb-1385-4440-8189-a3f804512956 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:14:22.617805 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-9a6edcbb-1385-4440-8189-a3f804512956 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 17 19:14:22.617805 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:14:22.617805 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 17 19:14:22.617805 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 17 19:14:22.617805 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 17 19:14:22.617805 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 17 19:14:22.617805 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 17 19:14:22.617805 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 17 19:14:22.617805 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 17 19:14:22.617805 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:14:22.617805 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 17 19:14:22.617805 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:14:22.617805 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:14:22.617805 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:14:22.617805 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:14:22.617805 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:14:22.617805 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:14:22.619051 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:14:22.619051 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:14:22.619051 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:14:22.619051 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:14:22.619051 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:14:22.619051 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:14:22.619051 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 17 19:14:22.619051 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_service(service_id) May 17 19:14:22.619051 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 17 19:14:22.619051 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 17 19:14:22.619051 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:14:22.619051 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:14:22.619051 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:14:22.619051 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:14:22.619051 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:14:22.619051 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:14:22.619994 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:14:22.619994 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:14:22.619994 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:14:22.619994 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:14:22.619994 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:14:22.619994 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:14:22.619994 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:14:22.619994 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:14:22.619994 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:14:22.619994 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:14:22.619994 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 17 19:14:22.619994 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 17 19:14:22.619994 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 17 19:14:22.619994 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:14:22.619994 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 645/1290] 10.222.0.58 () {56 vars in 1117 bytes} [Tue May 17 19:14:22 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:14:22.625030 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee47bb6-aff2-448e-bcb1-c56d338ce693 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:14:22.720592 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee47bb6-aff2-448e-bcb1-c56d338ce693 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:14:22.754493 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cc8f1a-6180-43ba-9a4e-2e4cf8bf47d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:14:22.755149 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cc8f1a-6180-43ba-9a4e-2e4cf8bf47d5 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:14:22.755425 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cc8f1a-6180-43ba-9a4e-2e4cf8bf47d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:14:22.755720 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cc8f1a-6180-43ba-9a4e-2e4cf8bf47d5 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:14:22.756436 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cc8f1a-6180-43ba-9a4e-2e4cf8bf47d5 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:14:22.756731 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cc8f1a-6180-43ba-9a4e-2e4cf8bf47d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:14:22.758094 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cc8f1a-6180-43ba-9a4e-2e4cf8bf47d5 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:14:22.763342 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 646/1291] 10.222.0.58 () {56 vars in 1116 bytes} [Tue May 17 19:14:22 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:14:22.769147 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6edcbb-1385-4440-8189-a3f804512956 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:14:22.873017 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6edcbb-1385-4440-8189-a3f804512956 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:14:22.908157 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3b3c70-d098-4f87-9928-e35edbb3b692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:14:22.908844 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3b3c70-d098-4f87-9928-e35edbb3b692 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:14:22.909120 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3b3c70-d098-4f87-9928-e35edbb3b692 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:14:22.909387 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3b3c70-d098-4f87-9928-e35edbb3b692 None admin] PATH_INFO: `/v3/services` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:14:22.910101 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3b3c70-d098-4f87-9928-e35edbb3b692 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:14:22.910400 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3b3c70-d098-4f87-9928-e35edbb3b692 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:14:22.911822 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3b3c70-d098-4f87-9928-e35edbb3b692 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:14:22.917080 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 646/1292] 10.222.0.58 () {56 vars in 1116 bytes} [Tue May 17 19:14:22 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:14:25.113362 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3419f0bc-5e8b-4a40-a846-e6f837c6ee43 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:14:25.113939 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3419f0bc-5e8b-4a40-a846-e6f837c6ee43 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:14:25.113939 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3419f0bc-5e8b-4a40-a846-e6f837c6ee43 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:14:25.115340 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 647/1293] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:14:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:14:25.121114 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2369a4-0895-497d-a39f-5d1e9e979a66 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:14:25.121529 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2369a4-0895-497d-a39f-5d1e9e979a66 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:14:25.121711 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2369a4-0895-497d-a39f-5d1e9e979a66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:14:25.138011 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-6a2369a4-0895-497d-a39f-5d1e9e979a66 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:14:25.139186 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-6a2369a4-0895-497d-a39f-5d1e9e979a66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:14:25.202459 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 647/1294] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:14:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 17 19:14:25.224480 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3419f0bc-5e8b-4a40-a846-e6f837c6ee43 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:14:25.320965 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3419f0bc-5e8b-4a40-a846-e6f837c6ee43 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:14:25.353418 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53811f7-da58-4554-8352-a730fee93097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:14:25.354594 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53811f7-da58-4554-8352-a730fee93097 None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:14:25.354865 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53811f7-da58-4554-8352-a730fee93097 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:14:25.355130 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53811f7-da58-4554-8352-a730fee93097 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:14:25.355973 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53811f7-da58-4554-8352-a730fee93097 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:14:25.357346 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53811f7-da58-4554-8352-a730fee93097 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:14:25.449846 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 648/1295] 10.222.0.58 () {60 vars in 1141 bytes} [Tue May 17 19:14:25 2022] POST /identity/v3/services => generated 246 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:14:27.647918 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16eb53e-aa5f-474f-8079-27fa3bf5edea None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:14:27.648488 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16eb53e-aa5f-474f-8079-27fa3bf5edea None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:14:27.648488 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16eb53e-aa5f-474f-8079-27fa3bf5edea None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:14:27.649844 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 648/1296] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:14:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:14:27.655531 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e8b2c2-aeb6-4778-8f67-8c38b2a2c36c None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:14:27.655952 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e8b2c2-aeb6-4778-8f67-8c38b2a2c36c None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:14:27.656068 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e8b2c2-aeb6-4778-8f67-8c38b2a2c36c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:14:27.673181 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-e8e8b2c2-aeb6-4778-8f67-8c38b2a2c36c None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:14:27.674329 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-e8e8b2c2-aeb6-4778-8f67-8c38b2a2c36c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:14:27.743375 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 649/1297] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:14:27 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 17 19:14:27.765458 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16eb53e-aa5f-474f-8079-27fa3bf5edea None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:14:27.864944 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16eb53e-aa5f-474f-8079-27fa3bf5edea None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:14:27.897696 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c567285-53cd-4800-a90a-5fd221a33e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:14:27.898521 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c567285-53cd-4800-a90a-5fd221a33e27 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:14:27.898747 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c567285-53cd-4800-a90a-5fd221a33e27 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:14:27.899011 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c567285-53cd-4800-a90a-5fd221a33e27 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:14:27.899672 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c567285-53cd-4800-a90a-5fd221a33e27 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:14:27.901054 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c567285-53cd-4800-a90a-5fd221a33e27 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:14:27.912537 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-8c567285-53cd-4800-a90a-5fd221a33e27 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 17 19:14:27.912537 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:14:27.912537 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 17 19:14:27.912537 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 17 19:14:27.912537 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 17 19:14:27.912537 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 17 19:14:27.912537 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 17 19:14:27.912537 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 17 19:14:27.912537 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 17 19:14:27.912537 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:14:27.912537 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 17 19:14:27.912537 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:14:27.912537 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:14:27.912537 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:14:27.912537 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:14:27.912537 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:14:27.912537 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:14:27.913621 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:14:27.913621 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:14:27.913621 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:14:27.913621 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:14:27.913621 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:14:27.913621 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:14:27.913621 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 17 19:14:27.913621 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_service(service_id) May 17 19:14:27.913621 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 17 19:14:27.913621 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 17 19:14:27.913621 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:14:27.913621 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:14:27.913621 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:14:27.913621 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:14:27.913621 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:14:27.913621 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:14:27.914529 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:14:27.914529 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:14:27.914529 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:14:27.914529 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:14:27.914529 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:14:27.914529 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:14:27.914529 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:14:27.914529 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:14:27.914529 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:14:27.914529 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:14:27.914529 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 17 19:14:27.914529 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 17 19:14:27.914529 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 17 19:14:27.914529 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:14:27.914529 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 649/1298] 10.222.0.58 () {56 vars in 1117 bytes} [Tue May 17 19:14:27 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:14:27.920233 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e8b2c2-aeb6-4778-8f67-8c38b2a2c36c None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:14:28.014346 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e8b2c2-aeb6-4778-8f67-8c38b2a2c36c None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:14:28.046308 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d21fbe-85b1-4bdc-a70c-15e9e6154cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:14:28.046960 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d21fbe-85b1-4bdc-a70c-15e9e6154cb4 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:14:28.047238 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d21fbe-85b1-4bdc-a70c-15e9e6154cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:14:28.047530 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d21fbe-85b1-4bdc-a70c-15e9e6154cb4 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:14:28.048220 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d21fbe-85b1-4bdc-a70c-15e9e6154cb4 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:14:28.048519 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d21fbe-85b1-4bdc-a70c-15e9e6154cb4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:14:28.049877 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d21fbe-85b1-4bdc-a70c-15e9e6154cb4 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:14:28.054766 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 650/1299] 10.222.0.58 () {56 vars in 1116 bytes} [Tue May 17 19:14:27 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:14:28.061235 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c567285-53cd-4800-a90a-5fd221a33e27 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:14:28.156134 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c567285-53cd-4800-a90a-5fd221a33e27 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:14:28.188479 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e004dfe4-cddf-427d-a534-d6ab5503e122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:14:28.189116 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e004dfe4-cddf-427d-a534-d6ab5503e122 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:14:28.189396 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e004dfe4-cddf-427d-a534-d6ab5503e122 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:14:28.189667 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e004dfe4-cddf-427d-a534-d6ab5503e122 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:14:28.190445 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e004dfe4-cddf-427d-a534-d6ab5503e122 None admin] RBAC: Adding query filter params (interface=public, service_id=3b3601a28ce3487fbb109a4910813260, region_id=RegionOne) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:14:28.190747 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e004dfe4-cddf-427d-a534-d6ab5503e122 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:14:28.192135 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e004dfe4-cddf-427d-a534-d6ab5503e122 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:14:28.196890 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 650/1300] 10.222.0.58 () {56 vars in 1251 bytes} [Tue May 17 19:14:28 2022] GET /identity/v3/endpoints?service_id=3b3601a28ce3487fbb109a4910813260&interface=public®ion_id=RegionOne => generated 194 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:14:28.202737 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d21fbe-85b1-4bdc-a70c-15e9e6154cb4 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:14:28.299012 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d21fbe-85b1-4bdc-a70c-15e9e6154cb4 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:14:28.330779 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf17225-012b-4fa5-893a-f61d24f61eec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:14:28.331426 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf17225-012b-4fa5-893a-f61d24f61eec None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:14:28.331731 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf17225-012b-4fa5-893a-f61d24f61eec None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:14:28.332003 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf17225-012b-4fa5-893a-f61d24f61eec None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:14:28.332668 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf17225-012b-4fa5-893a-f61d24f61eec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:14:28.334023 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf17225-012b-4fa5-893a-f61d24f61eec None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:14:28.341065 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 651/1301] 10.222.0.58 () {56 vars in 1087 bytes} [Tue May 17 19:14:28 2022] GET /identity/v3/services => generated 2191 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:14:30.517453 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf05ae91-2777-47db-8fdc-f16ea3273738 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:14:30.518063 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf05ae91-2777-47db-8fdc-f16ea3273738 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:14:30.518063 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf05ae91-2777-47db-8fdc-f16ea3273738 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:14:30.519390 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 651/1302] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:14:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:14:30.524475 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40952dac-9d9d-4f46-bd66-5924ab602af4 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:14:30.524917 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40952dac-9d9d-4f46-bd66-5924ab602af4 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:14:30.525010 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40952dac-9d9d-4f46-bd66-5924ab602af4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:14:30.542523 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-40952dac-9d9d-4f46-bd66-5924ab602af4 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:14:30.543724 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-40952dac-9d9d-4f46-bd66-5924ab602af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:14:30.606797 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 652/1303] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:14:30 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 17 19:14:30.629390 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf05ae91-2777-47db-8fdc-f16ea3273738 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:14:30.727469 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf05ae91-2777-47db-8fdc-f16ea3273738 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:14:30.759163 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41188d92-3133-47de-8087-57c5128f6bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:14:30.759973 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41188d92-3133-47de-8087-57c5128f6bfe None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:14:30.760832 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41188d92-3133-47de-8087-57c5128f6bfe None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:14:30.761132 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41188d92-3133-47de-8087-57c5128f6bfe None admin] PATH_INFO: `/v3/services/placement` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:14:30.761768 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41188d92-3133-47de-8087-57c5128f6bfe None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:14:30.763120 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41188d92-3133-47de-8087-57c5128f6bfe None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:14:30.774617 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-41188d92-3133-47de-8087-57c5128f6bfe None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 17 19:14:30.774617 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:14:30.774617 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 17 19:14:30.774617 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 17 19:14:30.774617 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 17 19:14:30.774617 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 17 19:14:30.774617 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 17 19:14:30.774617 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 17 19:14:30.774617 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 17 19:14:30.774617 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:14:30.774617 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 17 19:14:30.774617 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:14:30.774617 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): May 17 19:14:30.774617 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 17 19:14:30.774617 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 17 19:14:30.774617 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 17 19:14:30.774617 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 17 19:14:30.783886 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 17 19:14:30.783886 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 17 19:14:30.783886 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 17 19:14:30.783886 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 17 19:14:30.783886 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 17 19:14:30.783886 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 17 19:14:30.783886 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 17 19:14:30.783886 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_service(service_id) May 17 19:14:30.783886 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 17 19:14:30.783886 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 17 19:14:30.783886 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 17 19:14:30.783886 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 17 19:14:30.783886 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 17 19:14:30.783886 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 17 19:14:30.783886 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 17 19:14:30.783886 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( May 17 19:14:30.785011 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 17 19:14:30.785011 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( May 17 19:14:30.785011 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 17 19:14:30.785011 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() May 17 19:14:30.785011 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 17 19:14:30.785011 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 17 19:14:30.785011 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 17 19:14:30.785011 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() May 17 19:14:30.785011 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 17 19:14:30.785011 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( May 17 19:14:30.785011 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 17 19:14:30.785011 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 17 19:14:30.785011 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 17 19:14:30.785011 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application May 17 19:14:30.785011 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 652/1304] 10.222.0.58 () {56 vars in 1117 bytes} [Tue May 17 19:14:30 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 17 19:14:30.786008 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40952dac-9d9d-4f46-bd66-5924ab602af4 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:14:30.876019 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40952dac-9d9d-4f46-bd66-5924ab602af4 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:14:30.908123 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0132df85-aaa3-46be-9794-4e0a7c12658d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:14:30.908765 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0132df85-aaa3-46be-9794-4e0a7c12658d None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:14:30.909037 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0132df85-aaa3-46be-9794-4e0a7c12658d None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:14:30.909300 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0132df85-aaa3-46be-9794-4e0a7c12658d None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:14:30.910037 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0132df85-aaa3-46be-9794-4e0a7c12658d None admin] RBAC: Adding query filter params (name=placement) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:14:30.910331 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0132df85-aaa3-46be-9794-4e0a7c12658d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:14:30.911707 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0132df85-aaa3-46be-9794-4e0a7c12658d None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:14:30.917198 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 653/1305] 10.222.0.58 () {56 vars in 1116 bytes} [Tue May 17 19:14:30 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:14:30.922944 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41188d92-3133-47de-8087-57c5128f6bfe None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:14:31.018573 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41188d92-3133-47de-8087-57c5128f6bfe None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:14:31.050320 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20cb51d-cc99-47c8-a731-edc33e298c54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:14:31.050896 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20cb51d-cc99-47c8-a731-edc33e298c54 None admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:14:31.051170 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20cb51d-cc99-47c8-a731-edc33e298c54 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:14:31.051438 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20cb51d-cc99-47c8-a731-edc33e298c54 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:14:31.052190 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20cb51d-cc99-47c8-a731-edc33e298c54 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:14:31.053607 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20cb51d-cc99-47c8-a731-edc33e298c54 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:14:31.168950 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 653/1306] 10.222.0.58 () {60 vars in 1144 bytes} [Tue May 17 19:14:30 2022] POST /identity/v3/endpoints => generated 331 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:20:48.085080 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4381ea80-c171-40d3-a565-10b60a65a371 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:20:48.085988 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4381ea80-c171-40d3-a565-10b60a65a371 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:20:48.086408 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4381ea80-c171-40d3-a565-10b60a65a371 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:20:48.099074 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399591b9-f90e-44a5-810b-227175ca701d None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:20:48.101645 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399591b9-f90e-44a5-810b-227175ca701d None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:20:48.102624 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399591b9-f90e-44a5-810b-227175ca701d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:20:48.109809 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 654/1307] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:20:48 2022] GET /identity => generated 270 bytes in 25 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:20:48.295903 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-399591b9-f90e-44a5-810b-227175ca701d None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:20:48.298996 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-399591b9-f90e-44a5-810b-227175ca701d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:20:48.423906 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 654/1308] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:20:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:20:48.473028 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343d3ff5-11c8-4aac-b8d8-d64f0069a1aa None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:20:48.473440 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343d3ff5-11c8-4aac-b8d8-d64f0069a1aa None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:20:48.473797 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343d3ff5-11c8-4aac-b8d8-d64f0069a1aa None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:20:48.475780 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 655/1309] 10.222.0.58 () {54 vars in 964 bytes} [Tue May 17 19:20:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:20:48.482318 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4ff994-0abb-4f21-bd3d-f2647ce973e4 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:20:48.482928 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4ff994-0abb-4f21-bd3d-f2647ce973e4 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:20:48.484177 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4ff994-0abb-4f21-bd3d-f2647ce973e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:20:48.579250 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-aa4ff994-0abb-4f21-bd3d-f2647ce973e4 None None] MFA Rules not processed for user `1ac193d15f48463c98ff1d97770ed794`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:20:48.580643 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-aa4ff994-0abb-4f21-bd3d-f2647ce973e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:20:48.662093 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 655/1310] 10.222.0.58 () {58 vars in 1061 bytes} [Tue May 17 19:20:48 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:20:48.670947 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343d3ff5-11c8-4aac-b8d8-d64f0069a1aa None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:20:48.672997 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-343d3ff5-11c8-4aac-b8d8-d64f0069a1aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:20:48.782212 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343d3ff5-11c8-4aac-b8d8-d64f0069a1aa None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:20:48.859824 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0ce097-682c-4342-b8bc-0d2f9b1027b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:20:48.860755 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0ce097-682c-4342-b8bc-0d2f9b1027b9 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:20:48.861201 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0ce097-682c-4342-b8bc-0d2f9b1027b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:20:48.861630 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0ce097-682c-4342-b8bc-0d2f9b1027b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:20:48.863416 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-1a0ce097-682c-4342-b8bc-0d2f9b1027b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:20:48.986871 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0ce097-682c-4342-b8bc-0d2f9b1027b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:20:48.988358 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0ce097-682c-4342-b8bc-0d2f9b1027b9 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:20:49.008773 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 656/1311] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:20:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:20:51.580908 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7404e9ad-7606-478b-ba37-162993da315a None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:20:51.581476 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7404e9ad-7606-478b-ba37-162993da315a None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:20:51.581476 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7404e9ad-7606-478b-ba37-162993da315a None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:20:51.582884 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 656/1312] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:20:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:20:51.588297 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63c5170-2219-499c-befa-629ee05c0544 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:20:51.588772 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63c5170-2219-499c-befa-629ee05c0544 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:20:51.588875 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63c5170-2219-499c-befa-629ee05c0544 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:20:51.659786 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-e63c5170-2219-499c-befa-629ee05c0544 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:20:51.660995 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-e63c5170-2219-499c-befa-629ee05c0544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:20:51.758588 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 657/1313] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:20:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:20:51.796597 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7404e9ad-7606-478b-ba37-162993da315a None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:20:51.798192 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-7404e9ad-7606-478b-ba37-162993da315a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:20:51.899778 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7404e9ad-7606-478b-ba37-162993da315a None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:20:51.970599 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebce2b1b-36e0-451e-a045-53764194ee5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:20:51.971239 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebce2b1b-36e0-451e-a045-53764194ee5c service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:20:51.971545 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebce2b1b-36e0-451e-a045-53764194ee5c service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:20:51.971820 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebce2b1b-36e0-451e-a045-53764194ee5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:20:51.973438 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-ebce2b1b-36e0-451e-a045-53764194ee5c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:20:52.083873 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebce2b1b-36e0-451e-a045-53764194ee5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:20:52.085314 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebce2b1b-36e0-451e-a045-53764194ee5c service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:20:52.106960 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 657/1314] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:20:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:20:54.898001 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbdcb9e-6b71-41c0-81bc-586f348c0f3d None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:20:54.898560 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbdcb9e-6b71-41c0-81bc-586f348c0f3d None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:20:54.898560 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbdcb9e-6b71-41c0-81bc-586f348c0f3d None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:20:54.900160 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 658/1315] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:20:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:20:54.905939 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdebd088-d7e8-4963-bdc9-7a4bc00a77b2 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:20:54.906320 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdebd088-d7e8-4963-bdc9-7a4bc00a77b2 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:20:54.906490 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdebd088-d7e8-4963-bdc9-7a4bc00a77b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:20:54.968584 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-cdebd088-d7e8-4963-bdc9-7a4bc00a77b2 None None] MFA Rules not processed for user `ed3e18e472cf4c8fb0e0f8e477b961b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:20:54.969780 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-cdebd088-d7e8-4963-bdc9-7a4bc00a77b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:20:55.060660 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 658/1316] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:20:54 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:20:55.099610 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbdcb9e-6b71-41c0-81bc-586f348c0f3d None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:20:55.101369 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-5cbdcb9e-6b71-41c0-81bc-586f348c0f3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:20:55.198859 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbdcb9e-6b71-41c0-81bc-586f348c0f3d None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:20:55.269174 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18e2d6d-8bc7-4ed8-8774-73f7654dd743 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:20:55.269803 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18e2d6d-8bc7-4ed8-8774-73f7654dd743 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:20:55.270077 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18e2d6d-8bc7-4ed8-8774-73f7654dd743 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:20:55.270344 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18e2d6d-8bc7-4ed8-8774-73f7654dd743 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:20:55.271969 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-e18e2d6d-8bc7-4ed8-8774-73f7654dd743 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:20:55.374836 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18e2d6d-8bc7-4ed8-8774-73f7654dd743 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:20:55.376323 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18e2d6d-8bc7-4ed8-8774-73f7654dd743 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:20:55.393845 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 659/1317] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:20:55 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:20:58.427380 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b33291-a818-4b2d-a239-d2e146c9a53c None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:20:58.427951 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b33291-a818-4b2d-a239-d2e146c9a53c None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:20:58.427951 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b33291-a818-4b2d-a239-d2e146c9a53c None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:20:58.429279 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 659/1318] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:20:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:20:58.435154 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dc43d7-5fc5-4ee0-b538-c7ed309a0c91 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:20:58.435576 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dc43d7-5fc5-4ee0-b538-c7ed309a0c91 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:20:58.435743 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dc43d7-5fc5-4ee0-b538-c7ed309a0c91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:20:58.499434 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-e5dc43d7-5fc5-4ee0-b538-c7ed309a0c91 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:20:58.500651 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-e5dc43d7-5fc5-4ee0-b538-c7ed309a0c91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:20:58.593005 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 660/1319] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:20:58 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:20:58.632025 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b33291-a818-4b2d-a239-d2e146c9a53c None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:20:58.633533 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-69b33291-a818-4b2d-a239-d2e146c9a53c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:20:58.733242 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b33291-a818-4b2d-a239-d2e146c9a53c None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:20:58.802327 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22de1f3e-51cb-446e-a9d9-3552017f9e55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:20:58.802990 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22de1f3e-51cb-446e-a9d9-3552017f9e55 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:20:58.803263 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22de1f3e-51cb-446e-a9d9-3552017f9e55 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:20:58.803556 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22de1f3e-51cb-446e-a9d9-3552017f9e55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:20:58.805152 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-22de1f3e-51cb-446e-a9d9-3552017f9e55 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:20:58.910268 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22de1f3e-51cb-446e-a9d9-3552017f9e55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:20:58.911717 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22de1f3e-51cb-446e-a9d9-3552017f9e55 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:20:58.929754 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 660/1320] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:20:58 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:21:02.086361 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce2a07a-2e98-4515-9875-b1d4bbc11f77 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:21:02.086980 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce2a07a-2e98-4515-9875-b1d4bbc11f77 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:21:02.086980 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce2a07a-2e98-4515-9875-b1d4bbc11f77 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:21:02.088337 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 661/1321] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:21:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:21:02.094185 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b51b52-8cf2-409d-b386-fa90aa95c18a None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:21:02.094606 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b51b52-8cf2-409d-b386-fa90aa95c18a None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:21:02.094733 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b51b52-8cf2-409d-b386-fa90aa95c18a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:21:02.160544 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-b3b51b52-8cf2-409d-b386-fa90aa95c18a None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:21:02.161753 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-b3b51b52-8cf2-409d-b386-fa90aa95c18a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:21:02.254062 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 661/1322] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:21:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:21:02.289339 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce2a07a-2e98-4515-9875-b1d4bbc11f77 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:21:02.290848 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-7ce2a07a-2e98-4515-9875-b1d4bbc11f77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:21:02.389006 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce2a07a-2e98-4515-9875-b1d4bbc11f77 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:21:02.458704 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191aa472-8ae4-43af-a807-7f2451246182 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:21:02.459342 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191aa472-8ae4-43af-a807-7f2451246182 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:21:02.459651 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191aa472-8ae4-43af-a807-7f2451246182 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:21:02.460144 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191aa472-8ae4-43af-a807-7f2451246182 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:21:02.461578 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-191aa472-8ae4-43af-a807-7f2451246182 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:21:02.568006 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191aa472-8ae4-43af-a807-7f2451246182 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:21:02.569475 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191aa472-8ae4-43af-a807-7f2451246182 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:21:02.587764 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 662/1323] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:21:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:21:04.844147 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df160de7-3e40-4513-889b-39e14c06b26d None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:21:04.844716 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df160de7-3e40-4513-889b-39e14c06b26d None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:21:04.844716 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df160de7-3e40-4513-889b-39e14c06b26d None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:21:04.846126 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 662/1324] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:21:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:21:04.851597 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf337e0-03f4-45e7-8c9b-bf319d2970a2 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:21:04.852033 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf337e0-03f4-45e7-8c9b-bf319d2970a2 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:21:04.852140 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf337e0-03f4-45e7-8c9b-bf319d2970a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:21:04.914089 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-5cf337e0-03f4-45e7-8c9b-bf319d2970a2 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:21:04.915275 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-5cf337e0-03f4-45e7-8c9b-bf319d2970a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:21:05.016042 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 663/1325] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:21:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:21:05.065135 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86ddd3a-9a39-4626-a754-d44da9230bc4 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:21:05.065503 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86ddd3a-9a39-4626-a754-d44da9230bc4 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:21:05.065841 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86ddd3a-9a39-4626-a754-d44da9230bc4 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:21:05.067271 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 663/1326] 10.222.0.58 () {54 vars in 964 bytes} [Tue May 17 19:21:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:21:05.073003 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461acfee-4b5e-4c99-86f5-52956f1685ea None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:21:05.073294 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461acfee-4b5e-4c99-86f5-52956f1685ea None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:21:05.074222 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461acfee-4b5e-4c99-86f5-52956f1685ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:21:05.145881 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-461acfee-4b5e-4c99-86f5-52956f1685ea None None] MFA Rules not processed for user `1ac193d15f48463c98ff1d97770ed794`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:21:05.146992 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-461acfee-4b5e-4c99-86f5-52956f1685ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:21:05.220317 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 664/1327] 10.222.0.58 () {58 vars in 1061 bytes} [Tue May 17 19:21:05 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:21:05.229557 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86ddd3a-9a39-4626-a754-d44da9230bc4 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:21:05.231198 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-c86ddd3a-9a39-4626-a754-d44da9230bc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:21:05.330091 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86ddd3a-9a39-4626-a754-d44da9230bc4 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:21:05.402856 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a479d9-e964-4695-a470-840618e3416f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:21:05.403407 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a479d9-e964-4695-a470-840618e3416f service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:21:05.403711 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a479d9-e964-4695-a470-840618e3416f service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:21:05.403974 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a479d9-e964-4695-a470-840618e3416f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:21:05.405571 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-66a479d9-e964-4695-a470-840618e3416f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:21:05.508435 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a479d9-e964-4695-a470-840618e3416f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:21:05.509888 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a479d9-e964-4695-a470-840618e3416f service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:21:05.527789 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 664/1328] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:21:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:21:07.989770 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441125b2-34b6-473c-82f5-726cfd320b1b None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:21:07.990270 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441125b2-34b6-473c-82f5-726cfd320b1b None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:21:07.990392 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441125b2-34b6-473c-82f5-726cfd320b1b None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:21:07.994647 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 665/1329] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:21:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:21:07.997428 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccda37c-13ff-4c9c-b54c-994a1d850fe1 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:21:07.997831 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccda37c-13ff-4c9c-b54c-994a1d850fe1 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:21:07.997973 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccda37c-13ff-4c9c-b54c-994a1d850fe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:21:08.062791 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-1ccda37c-13ff-4c9c-b54c-994a1d850fe1 None None] MFA Rules not processed for user `ed3e18e472cf4c8fb0e0f8e477b961b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:21:08.064003 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-1ccda37c-13ff-4c9c-b54c-994a1d850fe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:21:08.155991 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 665/1330] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:21:07 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:21:08.192777 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441125b2-34b6-473c-82f5-726cfd320b1b None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:21:08.194294 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-441125b2-34b6-473c-82f5-726cfd320b1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:21:08.293787 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441125b2-34b6-473c-82f5-726cfd320b1b None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:21:08.362750 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4723cef-dc75-42d2-a561-63a4c6e4293e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:21:08.363382 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4723cef-dc75-42d2-a561-63a4c6e4293e service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:21:08.363685 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4723cef-dc75-42d2-a561-63a4c6e4293e service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:21:08.363952 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4723cef-dc75-42d2-a561-63a4c6e4293e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:21:08.365837 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-e4723cef-dc75-42d2-a561-63a4c6e4293e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:21:08.469533 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4723cef-dc75-42d2-a561-63a4c6e4293e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:21:08.470964 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4723cef-dc75-42d2-a561-63a4c6e4293e service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:21:08.488972 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 666/1331] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:21:08 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:21:11.223630 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400e5521-fc16-4d78-a12e-309e0c728a0f None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:21:11.224145 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400e5521-fc16-4d78-a12e-309e0c728a0f None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:21:11.224268 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400e5521-fc16-4d78-a12e-309e0c728a0f None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:21:11.225804 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 666/1332] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:21:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:21:11.231710 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbebf047-dcb7-42c3-aef1-4d0679838b00 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:21:11.232157 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbebf047-dcb7-42c3-aef1-4d0679838b00 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:21:11.232252 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbebf047-dcb7-42c3-aef1-4d0679838b00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:21:11.296592 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-bbebf047-dcb7-42c3-aef1-4d0679838b00 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:21:11.297718 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-bbebf047-dcb7-42c3-aef1-4d0679838b00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:21:11.394612 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 667/1333] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:21:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:21:11.430003 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400e5521-fc16-4d78-a12e-309e0c728a0f None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:21:11.431532 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-400e5521-fc16-4d78-a12e-309e0c728a0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:21:11.539954 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400e5521-fc16-4d78-a12e-309e0c728a0f None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:21:11.612267 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929ffc66-a14b-409f-b9b1-f5c412479f73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:21:11.612838 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929ffc66-a14b-409f-b9b1-f5c412479f73 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:21:11.613114 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929ffc66-a14b-409f-b9b1-f5c412479f73 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:21:11.613383 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929ffc66-a14b-409f-b9b1-f5c412479f73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:21:11.614995 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-929ffc66-a14b-409f-b9b1-f5c412479f73 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:21:11.719899 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929ffc66-a14b-409f-b9b1-f5c412479f73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:21:11.721323 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929ffc66-a14b-409f-b9b1-f5c412479f73 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:21:11.738928 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 667/1334] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:21:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:21:13.982063 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c431d305-ebd4-4984-b72e-ad48f4fe65e7 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:21:13.982616 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c431d305-ebd4-4984-b72e-ad48f4fe65e7 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:21:13.982616 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c431d305-ebd4-4984-b72e-ad48f4fe65e7 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:21:13.983986 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 668/1335] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:21:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:21:13.989158 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acae1983-c7e6-4d68-80dd-9324bb2e05bf None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:21:13.989574 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acae1983-c7e6-4d68-80dd-9324bb2e05bf None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:21:13.989714 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acae1983-c7e6-4d68-80dd-9324bb2e05bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:21:14.052915 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-acae1983-c7e6-4d68-80dd-9324bb2e05bf None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:21:14.054105 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-acae1983-c7e6-4d68-80dd-9324bb2e05bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:21:14.148249 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 668/1336] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:21:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:21:14.190477 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c431d305-ebd4-4984-b72e-ad48f4fe65e7 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:21:14.192084 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c431d305-ebd4-4984-b72e-ad48f4fe65e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:21:14.290645 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c431d305-ebd4-4984-b72e-ad48f4fe65e7 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:21:14.360218 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29888097-26f7-4b4c-a1f7-3ee755791c51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:21:14.360849 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29888097-26f7-4b4c-a1f7-3ee755791c51 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:21:14.361128 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29888097-26f7-4b4c-a1f7-3ee755791c51 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:21:14.361404 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29888097-26f7-4b4c-a1f7-3ee755791c51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:21:14.362993 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-29888097-26f7-4b4c-a1f7-3ee755791c51 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:21:14.470610 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29888097-26f7-4b4c-a1f7-3ee755791c51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:21:14.472069 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29888097-26f7-4b4c-a1f7-3ee755791c51 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:21:14.490268 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 669/1337] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:21:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:21:17.314529 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2901bf-53ce-45bb-bfbf-dac94dd1c04b None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:21:17.315087 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2901bf-53ce-45bb-bfbf-dac94dd1c04b None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:21:17.315087 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2901bf-53ce-45bb-bfbf-dac94dd1c04b None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:21:17.316463 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 669/1338] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:21:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:21:17.321550 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb649a47-ad87-4c5b-997a-a2d2af6486a8 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:21:17.321987 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb649a47-ad87-4c5b-997a-a2d2af6486a8 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:21:17.322082 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb649a47-ad87-4c5b-997a-a2d2af6486a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:21:17.383892 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-eb649a47-ad87-4c5b-997a-a2d2af6486a8 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:21:17.385061 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-eb649a47-ad87-4c5b-997a-a2d2af6486a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:21:17.478074 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 670/1339] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:21:17 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:21:17.514847 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2901bf-53ce-45bb-bfbf-dac94dd1c04b None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:21:17.516540 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-bb2901bf-53ce-45bb-bfbf-dac94dd1c04b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:21:17.614649 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2901bf-53ce-45bb-bfbf-dac94dd1c04b None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:21:17.683177 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5942dffc-27d6-4fc9-8053-79f5ed8c406c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:21:17.683830 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5942dffc-27d6-4fc9-8053-79f5ed8c406c service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:21:17.684106 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5942dffc-27d6-4fc9-8053-79f5ed8c406c service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:21:17.684373 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5942dffc-27d6-4fc9-8053-79f5ed8c406c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:21:17.685976 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-5942dffc-27d6-4fc9-8053-79f5ed8c406c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:21:17.788534 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5942dffc-27d6-4fc9-8053-79f5ed8c406c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:21:17.789880 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5942dffc-27d6-4fc9-8053-79f5ed8c406c service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:21:17.806826 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 670/1340] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:21:17 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:21:22.181509 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e5dc64-78c2-4220-8e6f-0df20edee502 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:21:22.182087 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e5dc64-78c2-4220-8e6f-0df20edee502 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:21:22.182087 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e5dc64-78c2-4220-8e6f-0df20edee502 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:21:22.183442 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 671/1341] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:21:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:21:22.188988 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b47e6f-58d4-497a-93bd-8f31b122c075 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:21:22.189414 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b47e6f-58d4-497a-93bd-8f31b122c075 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:21:22.189657 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b47e6f-58d4-497a-93bd-8f31b122c075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:21:22.259645 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-51b47e6f-58d4-497a-93bd-8f31b122c075 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:21:22.261292 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-51b47e6f-58d4-497a-93bd-8f31b122c075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:21:22.361719 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 671/1342] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:21:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:21:22.400404 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e5dc64-78c2-4220-8e6f-0df20edee502 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:21:22.402529 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-71e5dc64-78c2-4220-8e6f-0df20edee502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:21:22.495991 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e5dc64-78c2-4220-8e6f-0df20edee502 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:21:22.499559 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b39d34-c5a8-4b58-88f0-4048f99e5ebc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:21:22.500237 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b39d34-c5a8-4b58-88f0-4048f99e5ebc service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:21:22.500515 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b39d34-c5a8-4b58-88f0-4048f99e5ebc service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:21:22.500785 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b39d34-c5a8-4b58-88f0-4048f99e5ebc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:21:22.509408 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b39d34-c5a8-4b58-88f0-4048f99e5ebc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:21:22.511463 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b39d34-c5a8-4b58-88f0-4048f99e5ebc service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:21:22.520630 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 672/1343] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:21:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:21:25.593433 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7be915-d316-45c5-bc66-07522cb8ca5a None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:21:25.593964 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7be915-d316-45c5-bc66-07522cb8ca5a None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:21:25.593964 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7be915-d316-45c5-bc66-07522cb8ca5a None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:21:25.595341 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 672/1344] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:21:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:21:25.600932 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac97f381-5d04-410d-8dd4-00582c8d53e0 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:21:25.601354 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac97f381-5d04-410d-8dd4-00582c8d53e0 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:21:25.601460 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac97f381-5d04-410d-8dd4-00582c8d53e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:21:25.628791 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-ac97f381-5d04-410d-8dd4-00582c8d53e0 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:21:25.629981 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-ac97f381-5d04-410d-8dd4-00582c8d53e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:21:25.705315 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 673/1345] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:21:25 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:21:25.743240 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7be915-d316-45c5-bc66-07522cb8ca5a None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:21:25.753799 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7be915-d316-45c5-bc66-07522cb8ca5a None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:21:25.757533 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27c09a7-8052-455d-8a55-5f4ff96d3afc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:21:25.758172 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27c09a7-8052-455d-8a55-5f4ff96d3afc service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:21:25.758454 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27c09a7-8052-455d-8a55-5f4ff96d3afc service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:21:25.758727 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27c09a7-8052-455d-8a55-5f4ff96d3afc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:21:25.768139 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27c09a7-8052-455d-8a55-5f4ff96d3afc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:21:25.769631 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27c09a7-8052-455d-8a55-5f4ff96d3afc service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:21:25.779593 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 673/1346] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:21:25 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:21:30.035657 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fab4e1-a7c5-4c67-a877-450092d9ad00 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:21:30.036160 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fab4e1-a7c5-4c67-a877-450092d9ad00 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:21:30.036253 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fab4e1-a7c5-4c67-a877-450092d9ad00 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:21:30.037700 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 674/1347] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:21:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:21:30.042751 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc301b6d-0e53-4829-a11e-afea28f74087 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:21:30.043170 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc301b6d-0e53-4829-a11e-afea28f74087 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:21:30.043322 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc301b6d-0e53-4829-a11e-afea28f74087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:21:30.061176 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-bc301b6d-0e53-4829-a11e-afea28f74087 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:21:30.062358 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-bc301b6d-0e53-4829-a11e-afea28f74087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:21:30.071827 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 674/1348] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:21:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:21:30.108115 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fab4e1-a7c5-4c67-a877-450092d9ad00 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:21:30.120442 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fab4e1-a7c5-4c67-a877-450092d9ad00 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:21:30.124276 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22744a9c-dcc5-41df-be08-9a488f744738 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:21:30.124917 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22744a9c-dcc5-41df-be08-9a488f744738 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:21:30.125193 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22744a9c-dcc5-41df-be08-9a488f744738 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:21:30.125465 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22744a9c-dcc5-41df-be08-9a488f744738 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:21:30.137039 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22744a9c-dcc5-41df-be08-9a488f744738 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:21:30.137182 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22744a9c-dcc5-41df-be08-9a488f744738 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:21:30.155873 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 675/1349] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:21:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:21:35.484145 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ca728d-7a3d-4340-a26e-1fbe463b13ee None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:21:35.485190 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ca728d-7a3d-4340-a26e-1fbe463b13ee None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:21:35.485190 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ca728d-7a3d-4340-a26e-1fbe463b13ee None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:21:35.486137 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 675/1350] 10.222.0.58 () {54 vars in 921 bytes} [Tue May 17 19:21:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:21:35.496364 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28384402-58e0-4c76-80e9-6a2845168932 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:21:35.496364 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28384402-58e0-4c76-80e9-6a2845168932 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:21:35.496364 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28384402-58e0-4c76-80e9-6a2845168932 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:21:35.610095 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-28384402-58e0-4c76-80e9-6a2845168932 None None] MFA Rules not processed for user `06538aab5f194af7b7c677d6b94aef2a`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:21:35.612172 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-28384402-58e0-4c76-80e9-6a2845168932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:21:35.696356 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 676/1351] 10.222.0.58 () {58 vars in 1018 bytes} [Tue May 17 19:21:35 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 17 19:21:40.046354 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85523c5c-74ad-439d-ac68-d3243ae6049c None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:21:40.046891 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85523c5c-74ad-439d-ac68-d3243ae6049c None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:21:40.047148 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85523c5c-74ad-439d-ac68-d3243ae6049c None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:21:40.048686 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 676/1352] 10.222.0.58 () {54 vars in 921 bytes} [Tue May 17 19:21:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:21:40.053896 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c810c3e6-3850-461e-bd4d-e7478456f7f0 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:21:40.054336 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c810c3e6-3850-461e-bd4d-e7478456f7f0 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:21:40.054677 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c810c3e6-3850-461e-bd4d-e7478456f7f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:21:40.074124 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-c810c3e6-3850-461e-bd4d-e7478456f7f0 None None] MFA Rules not processed for user `06538aab5f194af7b7c677d6b94aef2a`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:21:40.075469 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c810c3e6-3850-461e-bd4d-e7478456f7f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:21:40.084845 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 677/1353] 10.222.0.58 () {58 vars in 1018 bytes} [Tue May 17 19:21:40 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:21:41.575049 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34937689-019b-49dd-8eea-a9cea9ce73a5 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:21:41.575609 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34937689-019b-49dd-8eea-a9cea9ce73a5 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:21:41.575778 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34937689-019b-49dd-8eea-a9cea9ce73a5 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:21:41.577226 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 677/1354] 10.222.0.58 () {54 vars in 921 bytes} [Tue May 17 19:21:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:21:41.582235 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b827605e-f432-40f6-ba52-7c749548df90 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:21:41.582632 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b827605e-f432-40f6-ba52-7c749548df90 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:21:41.582775 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b827605e-f432-40f6-ba52-7c749548df90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:21:41.601925 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-b827605e-f432-40f6-ba52-7c749548df90 None None] MFA Rules not processed for user `06538aab5f194af7b7c677d6b94aef2a`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:21:41.603155 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b827605e-f432-40f6-ba52-7c749548df90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:21:41.611701 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 678/1355] 10.222.0.58 () {58 vars in 1018 bytes} [Tue May 17 19:21:41 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:21:45.460924 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7737f8c9-2268-4760-9865-c20b5be6e9d4 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:21:45.461482 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7737f8c9-2268-4760-9865-c20b5be6e9d4 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:21:45.461482 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7737f8c9-2268-4760-9865-c20b5be6e9d4 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:21:45.463557 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 678/1356] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:21:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:21:45.468724 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd96add1-c4da-40ee-b6e5-08070992ace7 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:21:45.469146 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd96add1-c4da-40ee-b6e5-08070992ace7 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:21:45.469269 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd96add1-c4da-40ee-b6e5-08070992ace7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:21:45.486909 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-bd96add1-c4da-40ee-b6e5-08070992ace7 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:21:45.488159 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-bd96add1-c4da-40ee-b6e5-08070992ace7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:21:45.496245 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 679/1357] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:21:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:21:45.544275 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b1d8e8-4399-4ef0-a29b-8abad55fb7b3 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:21:45.546163 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b1d8e8-4399-4ef0-a29b-8abad55fb7b3 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:21:45.546513 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b1d8e8-4399-4ef0-a29b-8abad55fb7b3 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:21:45.548014 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 679/1358] 10.222.0.58 () {54 vars in 961 bytes} [Tue May 17 19:21:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:21:45.553397 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3c78dd-96f8-4021-9855-4f0c6c3e1770 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:21:45.553682 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3c78dd-96f8-4021-9855-4f0c6c3e1770 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:21:45.553970 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3c78dd-96f8-4021-9855-4f0c6c3e1770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:21:45.601744 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-bf3c78dd-96f8-4021-9855-4f0c6c3e1770 None None] MFA Rules not processed for user `65c1f878b0dc4d63a4f5dcb12fac1dc1`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:21:45.603132 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-bf3c78dd-96f8-4021-9855-4f0c6c3e1770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:21:45.683967 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 680/1359] 10.222.0.58 () {58 vars in 1058 bytes} [Tue May 17 19:21:45 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:21:45.691908 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b1d8e8-4399-4ef0-a29b-8abad55fb7b3 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:21:45.707010 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b1d8e8-4399-4ef0-a29b-8abad55fb7b3 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:21:45.712842 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7f0c24-3c55-4886-9a78-eb8b10388b60 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c1f878b0dc4d63a4f5dcb12fac1dc1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:21:45.713498 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7f0c24-3c55-4886-9a78-eb8b10388b60 service nova] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:21:45.713895 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7f0c24-3c55-4886-9a78-eb8b10388b60 service nova] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:21:45.714271 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7f0c24-3c55-4886-9a78-eb8b10388b60 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:21:45.722432 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7f0c24-3c55-4886-9a78-eb8b10388b60 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:21:45.724366 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7f0c24-3c55-4886-9a78-eb8b10388b60 service nova] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:21:45.751689 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 680/1360] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:21:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:21:48.161522 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088566c9-7046-4a11-ac10-1f3eeb937c6d None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:21:48.162088 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088566c9-7046-4a11-ac10-1f3eeb937c6d None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:21:48.162088 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088566c9-7046-4a11-ac10-1f3eeb937c6d None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:21:48.163722 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 681/1361] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:21:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:21:48.168934 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596bf89c-2a21-4426-9cb6-419e0a8cedf4 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:21:48.169355 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596bf89c-2a21-4426-9cb6-419e0a8cedf4 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:21:48.169496 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596bf89c-2a21-4426-9cb6-419e0a8cedf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:21:48.187442 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-596bf89c-2a21-4426-9cb6-419e0a8cedf4 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:21:48.188665 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-596bf89c-2a21-4426-9cb6-419e0a8cedf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:21:48.196702 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 681/1362] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:21:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:21:48.239347 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088566c9-7046-4a11-ac10-1f3eeb937c6d None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:21:48.256083 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088566c9-7046-4a11-ac10-1f3eeb937c6d None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:21:48.262678 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc0b195-6240-456e-bc03-2deba5875ce3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c1f878b0dc4d63a4f5dcb12fac1dc1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:21:48.263367 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc0b195-6240-456e-bc03-2deba5875ce3 service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:21:48.263669 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc0b195-6240-456e-bc03-2deba5875ce3 service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:21:48.263944 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc0b195-6240-456e-bc03-2deba5875ce3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:21:48.273117 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc0b195-6240-456e-bc03-2deba5875ce3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:21:48.275450 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc0b195-6240-456e-bc03-2deba5875ce3 service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:21:48.288593 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 682/1363] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:21:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:21:50.930364 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ecfe5f-7ac1-4cc1-9a68-ea494b6d76be None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:21:50.931428 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ecfe5f-7ac1-4cc1-9a68-ea494b6d76be None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:21:50.931428 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ecfe5f-7ac1-4cc1-9a68-ea494b6d76be None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:21:50.933111 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 682/1364] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:21:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:21:50.939285 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7358139-2b88-4aff-ab66-0efe3d8cefd5 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:21:50.940122 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7358139-2b88-4aff-ab66-0efe3d8cefd5 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:21:50.940400 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7358139-2b88-4aff-ab66-0efe3d8cefd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:21:50.968065 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-a7358139-2b88-4aff-ab66-0efe3d8cefd5 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:21:50.969522 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a7358139-2b88-4aff-ab66-0efe3d8cefd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:21:50.978021 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 683/1365] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:21:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:21:51.024017 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1cd9ff-e536-44fd-9435-083433eba070 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:21:51.024379 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1cd9ff-e536-44fd-9435-083433eba070 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:21:51.024727 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1cd9ff-e536-44fd-9435-083433eba070 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:21:51.026180 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 683/1366] 10.222.0.58 () {54 vars in 961 bytes} [Tue May 17 19:21:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:21:51.031425 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbe2d16-e01c-47be-ad0a-446a510b5f21 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:21:51.031746 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbe2d16-e01c-47be-ad0a-446a510b5f21 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:21:51.032008 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbe2d16-e01c-47be-ad0a-446a510b5f21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:21:51.049388 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-3cbe2d16-e01c-47be-ad0a-446a510b5f21 None None] MFA Rules not processed for user `65c1f878b0dc4d63a4f5dcb12fac1dc1`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:21:51.050539 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-3cbe2d16-e01c-47be-ad0a-446a510b5f21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:21:51.060091 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 684/1367] 10.222.0.58 () {58 vars in 1058 bytes} [Tue May 17 19:21:51 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:21:51.067398 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1cd9ff-e536-44fd-9435-083433eba070 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:21:51.082245 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1cd9ff-e536-44fd-9435-083433eba070 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:21:51.087687 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ae5cc3-7df7-486c-b9b1-fe29c7be1b8e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c1f878b0dc4d63a4f5dcb12fac1dc1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:21:51.088317 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ae5cc3-7df7-486c-b9b1-fe29c7be1b8e service nova] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:21:51.088597 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ae5cc3-7df7-486c-b9b1-fe29c7be1b8e service nova] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:21:51.088871 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ae5cc3-7df7-486c-b9b1-fe29c7be1b8e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:21:51.096914 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ae5cc3-7df7-486c-b9b1-fe29c7be1b8e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:21:51.098472 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ae5cc3-7df7-486c-b9b1-fe29c7be1b8e service nova] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:21:51.108081 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 684/1368] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:21:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 17 19:21:53.673706 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54da8d8c-5d56-4173-98f0-fd913310871f None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:21:53.674326 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8093ed4f-c161-47b5-bfb7-d32343934ec7 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:21:53.674705 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8093ed4f-c161-47b5-bfb7-d32343934ec7 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:21:53.674999 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8093ed4f-c161-47b5-bfb7-d32343934ec7 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:21:53.675392 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54da8d8c-5d56-4173-98f0-fd913310871f None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:21:53.675724 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54da8d8c-5d56-4173-98f0-fd913310871f None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:21:53.676626 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 685/1369] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:21:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:21:53.678617 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 685/1370] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:21:53 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 17 19:21:53.682671 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a490aa91-8e94-4f42-8333-a4d534951805 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:21:53.683227 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a490aa91-8e94-4f42-8333-a4d534951805 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:21:53.683515 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a490aa91-8e94-4f42-8333-a4d534951805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:21:53.684505 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf02744-4a57-4713-9fb8-9d1c2064d34a None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:21:53.684861 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf02744-4a57-4713-9fb8-9d1c2064d34a None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:21:53.685211 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf02744-4a57-4713-9fb8-9d1c2064d34a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:21:53.702571 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-a490aa91-8e94-4f42-8333-a4d534951805 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:21:53.703739 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-ebf02744-4a57-4713-9fb8-9d1c2064d34a None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:21:53.703884 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-a490aa91-8e94-4f42-8333-a4d534951805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:21:53.704920 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-ebf02744-4a57-4713-9fb8-9d1c2064d34a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:21:53.712643 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 686/1371] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:21:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:21:53.717857 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 686/1372] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:21:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:21:53.756421 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a490aa91-8e94-4f42-8333-a4d534951805 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:21:53.769164 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a490aa91-8e94-4f42-8333-a4d534951805 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:21:53.775717 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70dc1a5-c9b4-497e-a4ee-fce73643b037 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c1f878b0dc4d63a4f5dcb12fac1dc1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:21:53.776366 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70dc1a5-c9b4-497e-a4ee-fce73643b037 service nova] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:21:53.776640 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70dc1a5-c9b4-497e-a4ee-fce73643b037 service nova] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:21:53.776907 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70dc1a5-c9b4-497e-a4ee-fce73643b037 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:21:53.785791 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70dc1a5-c9b4-497e-a4ee-fce73643b037 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:21:53.787373 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70dc1a5-c9b4-497e-a4ee-fce73643b037 service nova] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:21:53.796968 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 687/1373] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:21:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 17 19:21:54.024871 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc58272-ba27-481d-9866-715806ba1df7 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:21:54.024871 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc58272-ba27-481d-9866-715806ba1df7 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:21:54.024871 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc58272-ba27-481d-9866-715806ba1df7 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:21:54.025418 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 687/1374] 10.222.0.58 () {54 vars in 963 bytes} [Tue May 17 19:21:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:21:54.030747 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2d84ff-2a9b-4472-be28-5ab5415d34f6 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:21:54.031273 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2d84ff-2a9b-4472-be28-5ab5415d34f6 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:21:54.031594 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2d84ff-2a9b-4472-be28-5ab5415d34f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:21:54.085949 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-9b2d84ff-2a9b-4472-be28-5ab5415d34f6 None None] MFA Rules not processed for user `d05409b38ae346458abfe41ebda1c59e`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:21:54.087754 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-9b2d84ff-2a9b-4472-be28-5ab5415d34f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:21:54.161252 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 688/1375] 10.222.0.58 () {58 vars in 1060 bytes} [Tue May 17 19:21:54 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:21:54.162616 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc58272-ba27-481d-9866-715806ba1df7 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:21:54.173743 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc58272-ba27-481d-9866-715806ba1df7 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:21:54.177422 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc460742-5f61-43cd-8ba9-d9c273324381 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd05409b38ae346458abfe41ebda1c59e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:21:54.178145 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc460742-5f61-43cd-8ba9-d9c273324381 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:21:54.178428 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc460742-5f61-43cd-8ba9-d9c273324381 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:21:54.178701 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc460742-5f61-43cd-8ba9-d9c273324381 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:21:54.189169 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc460742-5f61-43cd-8ba9-d9c273324381 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:21:54.190823 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc460742-5f61-43cd-8ba9-d9c273324381 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:21:54.203797 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 688/1376] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:21:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:21:56.207117 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a49366e-e061-45d4-b242-9f4142fc8ab9 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:21:56.207643 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a49366e-e061-45d4-b242-9f4142fc8ab9 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:21:56.207643 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a49366e-e061-45d4-b242-9f4142fc8ab9 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:21:56.208991 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 689/1377] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:21:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:21:56.215027 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e55b61-5dca-4a65-aeca-a99d456f0f49 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:21:56.215454 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e55b61-5dca-4a65-aeca-a99d456f0f49 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:21:56.215716 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e55b61-5dca-4a65-aeca-a99d456f0f49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:21:56.234305 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-e3e55b61-5dca-4a65-aeca-a99d456f0f49 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:21:56.235517 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-e3e55b61-5dca-4a65-aeca-a99d456f0f49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:21:56.243968 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 689/1378] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:21:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:21:56.287785 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a49366e-e061-45d4-b242-9f4142fc8ab9 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:21:56.303078 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a49366e-e061-45d4-b242-9f4142fc8ab9 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:21:56.308710 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcaf963-1731-4437-90fc-eb66aa773f1b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c1f878b0dc4d63a4f5dcb12fac1dc1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:21:56.309344 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcaf963-1731-4437-90fc-eb66aa773f1b service nova] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:21:56.309624 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcaf963-1731-4437-90fc-eb66aa773f1b service nova] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:21:56.309900 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcaf963-1731-4437-90fc-eb66aa773f1b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:21:56.318843 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcaf963-1731-4437-90fc-eb66aa773f1b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:21:56.320457 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcaf963-1731-4437-90fc-eb66aa773f1b service nova] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:21:56.330725 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 690/1379] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:21:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 17 19:21:58.718878 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4c964d-25e3-4ee1-ba25-0fb940285123 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:21:58.719414 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4c964d-25e3-4ee1-ba25-0fb940285123 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:21:58.719414 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4c964d-25e3-4ee1-ba25-0fb940285123 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:21:58.720960 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 690/1380] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:21:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:21:58.726321 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7129e6c-9bb0-4dc2-8c19-d4e23f109e34 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:21:58.726735 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7129e6c-9bb0-4dc2-8c19-d4e23f109e34 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:21:58.726870 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7129e6c-9bb0-4dc2-8c19-d4e23f109e34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:21:58.744743 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-d7129e6c-9bb0-4dc2-8c19-d4e23f109e34 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:21:58.746019 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-d7129e6c-9bb0-4dc2-8c19-d4e23f109e34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:21:58.753863 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 691/1381] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:21:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:21:58.796911 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4c964d-25e3-4ee1-ba25-0fb940285123 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:21:58.811075 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4c964d-25e3-4ee1-ba25-0fb940285123 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:21:58.816475 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4f9423-65cd-4c18-ae17-c1b55dbf2add service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c1f878b0dc4d63a4f5dcb12fac1dc1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:21:58.817104 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4f9423-65cd-4c18-ae17-c1b55dbf2add service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:21:58.817381 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4f9423-65cd-4c18-ae17-c1b55dbf2add service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:21:58.817659 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4f9423-65cd-4c18-ae17-c1b55dbf2add service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:21:58.826488 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4f9423-65cd-4c18-ae17-c1b55dbf2add service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:21:58.828121 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4f9423-65cd-4c18-ae17-c1b55dbf2add service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:21:58.838162 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 691/1382] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:21:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:22:01.243678 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f855b829-6fce-449b-aaf6-19425bac2ff6 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:22:01.244208 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f855b829-6fce-449b-aaf6-19425bac2ff6 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:22:01.244208 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f855b829-6fce-449b-aaf6-19425bac2ff6 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:22:01.245504 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 692/1383] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:22:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:22:01.250875 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e6253c-2977-46e0-9c6f-1b5df66bfdd7 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:22:01.251270 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e6253c-2977-46e0-9c6f-1b5df66bfdd7 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:22:01.251429 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e6253c-2977-46e0-9c6f-1b5df66bfdd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:22:01.274315 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-d2e6253c-2977-46e0-9c6f-1b5df66bfdd7 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:22:01.275520 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d2e6253c-2977-46e0-9c6f-1b5df66bfdd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:22:01.283546 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 692/1384] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:22:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:22:01.325343 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f855b829-6fce-449b-aaf6-19425bac2ff6 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:22:01.340789 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f855b829-6fce-449b-aaf6-19425bac2ff6 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:22:01.346274 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f9920f-21a7-4f35-a6a7-21c00e12d9ed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c1f878b0dc4d63a4f5dcb12fac1dc1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:22:01.346913 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f9920f-21a7-4f35-a6a7-21c00e12d9ed service nova] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:22:01.347201 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f9920f-21a7-4f35-a6a7-21c00e12d9ed service nova] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:22:01.347472 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f9920f-21a7-4f35-a6a7-21c00e12d9ed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:22:01.356940 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f9920f-21a7-4f35-a6a7-21c00e12d9ed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:22:01.358556 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f9920f-21a7-4f35-a6a7-21c00e12d9ed service nova] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:22:01.371661 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 693/1385] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:22:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:22:03.584510 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eae81bd-cc7b-46bb-9b9d-ccb091edb144 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:22:03.585102 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eae81bd-cc7b-46bb-9b9d-ccb091edb144 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:22:03.585102 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eae81bd-cc7b-46bb-9b9d-ccb091edb144 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:22:03.586642 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 693/1386] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:22:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:22:03.591829 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc3e0bd-09b8-4f3b-a957-9cbeaca159b1 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:22:03.592409 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc3e0bd-09b8-4f3b-a957-9cbeaca159b1 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:22:03.592409 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc3e0bd-09b8-4f3b-a957-9cbeaca159b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:22:03.610982 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-8cc3e0bd-09b8-4f3b-a957-9cbeaca159b1 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:22:03.612269 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-8cc3e0bd-09b8-4f3b-a957-9cbeaca159b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:22:03.620900 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 694/1387] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:22:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:22:03.665051 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eae81bd-cc7b-46bb-9b9d-ccb091edb144 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:22:03.678984 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eae81bd-cc7b-46bb-9b9d-ccb091edb144 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:22:03.684477 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b603fc5a-b4ba-418d-8ef9-9fbe0d280e8b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c1f878b0dc4d63a4f5dcb12fac1dc1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:22:03.685105 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b603fc5a-b4ba-418d-8ef9-9fbe0d280e8b service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:22:03.685382 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b603fc5a-b4ba-418d-8ef9-9fbe0d280e8b service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:22:03.685655 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b603fc5a-b4ba-418d-8ef9-9fbe0d280e8b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:22:03.693511 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b603fc5a-b4ba-418d-8ef9-9fbe0d280e8b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:22:03.695078 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b603fc5a-b4ba-418d-8ef9-9fbe0d280e8b service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:22:03.704465 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 694/1388] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:22:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:22:06.150858 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f59b104-0f48-44d4-92f9-72a82e21dca1 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:22:06.151438 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f59b104-0f48-44d4-92f9-72a82e21dca1 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:22:06.151438 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f59b104-0f48-44d4-92f9-72a82e21dca1 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:22:06.152891 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 695/1389] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:22:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:22:06.159245 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3fcbde-4bc7-4c7b-898d-07c888ec7e28 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:22:06.159718 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3fcbde-4bc7-4c7b-898d-07c888ec7e28 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:22:06.159811 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3fcbde-4bc7-4c7b-898d-07c888ec7e28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:22:06.177661 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-bf3fcbde-4bc7-4c7b-898d-07c888ec7e28 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:22:06.178850 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-bf3fcbde-4bc7-4c7b-898d-07c888ec7e28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:22:06.187194 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 695/1390] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:22:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:22:06.230412 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f59b104-0f48-44d4-92f9-72a82e21dca1 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:22:06.245049 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f59b104-0f48-44d4-92f9-72a82e21dca1 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:22:06.250499 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ff903a-731a-4435-beeb-6c698c88e035 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c1f878b0dc4d63a4f5dcb12fac1dc1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:22:06.251125 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ff903a-731a-4435-beeb-6c698c88e035 service nova] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:22:06.251405 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ff903a-731a-4435-beeb-6c698c88e035 service nova] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:22:06.251709 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ff903a-731a-4435-beeb-6c698c88e035 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:22:06.260352 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ff903a-731a-4435-beeb-6c698c88e035 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:22:06.261926 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ff903a-731a-4435-beeb-6c698c88e035 service nova] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:22:06.271324 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 696/1391] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:22:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:22:09.024881 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ed7597-304e-4deb-8f6a-0d2c01d3c422 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:22:09.025492 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ed7597-304e-4deb-8f6a-0d2c01d3c422 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:22:09.025492 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ed7597-304e-4deb-8f6a-0d2c01d3c422 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:22:09.026784 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 696/1392] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:22:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:22:09.032355 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a74e814-9c08-4a53-af49-fbf78c6e6fab None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:22:09.032806 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a74e814-9c08-4a53-af49-fbf78c6e6fab None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:22:09.032909 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a74e814-9c08-4a53-af49-fbf78c6e6fab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:22:09.052123 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-5a74e814-9c08-4a53-af49-fbf78c6e6fab None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:22:09.053367 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-5a74e814-9c08-4a53-af49-fbf78c6e6fab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:22:09.061639 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 697/1393] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:22:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 17 19:22:09.106065 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ed7597-304e-4deb-8f6a-0d2c01d3c422 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:22:09.120151 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ed7597-304e-4deb-8f6a-0d2c01d3c422 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:22:09.125405 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1601b0e0-6775-4f09-87fe-43b39d940a16 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c1f878b0dc4d63a4f5dcb12fac1dc1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:22:09.126022 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1601b0e0-6775-4f09-87fe-43b39d940a16 service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:22:09.126299 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1601b0e0-6775-4f09-87fe-43b39d940a16 service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:22:09.126565 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1601b0e0-6775-4f09-87fe-43b39d940a16 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:22:09.137473 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1601b0e0-6775-4f09-87fe-43b39d940a16 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:22:09.139077 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1601b0e0-6775-4f09-87fe-43b39d940a16 service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:22:09.149686 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 697/1394] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:22:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:22:10.047955 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ca0997-872d-435f-8cdb-d83e12c08013 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:22:10.048430 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ca0997-872d-435f-8cdb-d83e12c08013 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:22:10.048536 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ca0997-872d-435f-8cdb-d83e12c08013 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:22:10.050088 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 698/1395] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:22:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:22:10.055655 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef99a74-4e19-49a5-b4d3-9ce8bcaa8db6 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:22:10.056088 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef99a74-4e19-49a5-b4d3-9ce8bcaa8db6 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:22:10.056205 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef99a74-4e19-49a5-b4d3-9ce8bcaa8db6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:22:10.073298 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-aef99a74-4e19-49a5-b4d3-9ce8bcaa8db6 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:22:10.074465 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-aef99a74-4e19-49a5-b4d3-9ce8bcaa8db6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:22:10.082131 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 698/1396] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:22:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:22:10.137132 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae67c28-542a-4c36-b95a-1d54aa47f50a None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:22:10.137489 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae67c28-542a-4c36-b95a-1d54aa47f50a None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:22:10.137829 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae67c28-542a-4c36-b95a-1d54aa47f50a None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:22:10.139270 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 699/1397] 10.222.0.58 () {54 vars in 962 bytes} [Tue May 17 19:22:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:22:10.144954 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bad084d-05a8-4f28-a7c6-471ebbd376e2 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:22:10.145239 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bad084d-05a8-4f28-a7c6-471ebbd376e2 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:22:10.145511 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bad084d-05a8-4f28-a7c6-471ebbd376e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:22:10.193168 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-9bad084d-05a8-4f28-a7c6-471ebbd376e2 None None] MFA Rules not processed for user `57e888884d2b4629bfd12d1034ce5be4`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:22:10.194365 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-9bad084d-05a8-4f28-a7c6-471ebbd376e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:22:10.252243 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 699/1398] 10.222.0.58 () {58 vars in 1059 bytes} [Tue May 17 19:22:10 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:22:10.259837 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae67c28-542a-4c36-b95a-1d54aa47f50a None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:22:10.270169 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae67c28-542a-4c36-b95a-1d54aa47f50a None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:22:10.273662 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372506b2-f23b-4c23-8393-0abe1a09c6dd service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57e888884d2b4629bfd12d1034ce5be4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:22:10.274287 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372506b2-f23b-4c23-8393-0abe1a09c6dd service glance] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:22:10.274568 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372506b2-f23b-4c23-8393-0abe1a09c6dd service glance] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:22:10.275108 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372506b2-f23b-4c23-8393-0abe1a09c6dd service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:22:10.283715 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372506b2-f23b-4c23-8393-0abe1a09c6dd service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:22:10.285129 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372506b2-f23b-4c23-8393-0abe1a09c6dd service glance] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:22:10.294644 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 700/1399] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:22:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:22:10.868561 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6bfcd5-0288-4c3b-bfe0-a960ba7613bb None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:22:10.868851 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6bfcd5-0288-4c3b-bfe0-a960ba7613bb None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:22:10.869127 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6bfcd5-0288-4c3b-bfe0-a960ba7613bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:22:10.916125 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-7e6bfcd5-0288-4c3b-bfe0-a960ba7613bb None None] MFA Rules not processed for user `2b90b57c542440448579b3aadda8b241`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:22:10.917302 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-7e6bfcd5-0288-4c3b-bfe0-a960ba7613bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:22:10.979832 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 700/1400] 10.222.0.58 () {58 vars in 1009 bytes} [Tue May 17 19:22:10 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:22:11.021206 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d14356-2ec3-4083-8464-f1cc759b5cef None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:22:11.021593 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d14356-2ec3-4083-8464-f1cc759b5cef None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:22:11.021930 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d14356-2ec3-4083-8464-f1cc759b5cef None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:22:11.023464 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 701/1401] 10.222.0.58 () {54 vars in 942 bytes} [Tue May 17 19:22:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:22:11.030806 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebcb239-662e-4f75-9e74-1288eaa18eec None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:22:11.031093 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebcb239-662e-4f75-9e74-1288eaa18eec None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:22:11.031369 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebcb239-662e-4f75-9e74-1288eaa18eec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:22:11.078387 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-4ebcb239-662e-4f75-9e74-1288eaa18eec None None] MFA Rules not processed for user `b0cb292002434d30bc81f370d120a2a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:22:11.079609 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-4ebcb239-662e-4f75-9e74-1288eaa18eec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:22:11.149311 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 701/1402] 10.222.0.58 () {58 vars in 1039 bytes} [Tue May 17 19:22:11 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:22:11.158615 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d14356-2ec3-4083-8464-f1cc759b5cef None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:22:11.172025 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d14356-2ec3-4083-8464-f1cc759b5cef None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:22:11.177160 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f1f1e9-8f27-4067-9959-d20c60131152 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0cb292002434d30bc81f370d120a2a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:22:11.177794 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f1f1e9-8f27-4067-9959-d20c60131152 service swift] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:22:11.178072 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f1f1e9-8f27-4067-9959-d20c60131152 service swift] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:22:11.178348 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f1f1e9-8f27-4067-9959-d20c60131152 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:22:11.186911 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f1f1e9-8f27-4067-9959-d20c60131152 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:22:11.188539 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f1f1e9-8f27-4067-9959-d20c60131152 service swift] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:22:11.197293 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 702/1403] 10.222.0.58 () {60 vars in 1384 bytes} [Tue May 17 19:22:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 17 19:22:11.553563 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4177e964-22cd-402e-bedd-b6daf1a8e741 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:22:11.553841 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4177e964-22cd-402e-bedd-b6daf1a8e741 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:22:11.554115 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4177e964-22cd-402e-bedd-b6daf1a8e741 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:22:11.556229 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 702/1404] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:22:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:22:11.561493 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab49a50-b772-4708-9fa8-ed5636a627e8 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:22:11.561751 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab49a50-b772-4708-9fa8-ed5636a627e8 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:22:11.562055 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab49a50-b772-4708-9fa8-ed5636a627e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:22:11.579569 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-3ab49a50-b772-4708-9fa8-ed5636a627e8 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:22:11.580762 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-3ab49a50-b772-4708-9fa8-ed5636a627e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:22:11.588629 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 703/1405] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:22:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 17 19:22:11.634576 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4177e964-22cd-402e-bedd-b6daf1a8e741 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:22:11.652204 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4177e964-22cd-402e-bedd-b6daf1a8e741 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:22:11.657784 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc0912b-7399-4c6a-89c0-6d9a272880a1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c1f878b0dc4d63a4f5dcb12fac1dc1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:22:11.658425 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc0912b-7399-4c6a-89c0-6d9a272880a1 service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:22:11.658725 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc0912b-7399-4c6a-89c0-6d9a272880a1 service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:22:11.658968 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc0912b-7399-4c6a-89c0-6d9a272880a1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:22:11.667899 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc0912b-7399-4c6a-89c0-6d9a272880a1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:22:11.669651 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc0912b-7399-4c6a-89c0-6d9a272880a1 service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:22:11.682995 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 703/1406] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:22:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:22:14.140023 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64f3ea5-f765-446c-9c8a-feee253e70bb None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:22:14.140551 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64f3ea5-f765-446c-9c8a-feee253e70bb None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:22:14.140551 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64f3ea5-f765-446c-9c8a-feee253e70bb None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:22:14.141940 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 704/1407] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:22:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:22:14.147596 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fafa5bf-78de-43f7-9140-c0b2698581d6 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:22:14.148024 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fafa5bf-78de-43f7-9140-c0b2698581d6 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:22:14.148171 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fafa5bf-78de-43f7-9140-c0b2698581d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:22:14.165977 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-1fafa5bf-78de-43f7-9140-c0b2698581d6 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:22:14.167348 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-1fafa5bf-78de-43f7-9140-c0b2698581d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:22:14.176225 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 704/1408] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:22:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:22:14.220754 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64f3ea5-f765-446c-9c8a-feee253e70bb None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:22:14.238528 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64f3ea5-f765-446c-9c8a-feee253e70bb None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:22:14.245388 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ab3a57-afb2-4497-8e3b-9744fe1581b1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c1f878b0dc4d63a4f5dcb12fac1dc1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:22:14.246035 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ab3a57-afb2-4497-8e3b-9744fe1581b1 service nova] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:22:14.246310 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ab3a57-afb2-4497-8e3b-9744fe1581b1 service nova] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:22:14.246580 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ab3a57-afb2-4497-8e3b-9744fe1581b1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:22:14.255050 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ab3a57-afb2-4497-8e3b-9744fe1581b1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:22:14.256647 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ab3a57-afb2-4497-8e3b-9744fe1581b1 service nova] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:22:14.266706 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 705/1409] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:22:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:22:35.472558 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dcc56c-69f1-4b05-bc26-ed66a36880e1 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:22:35.473193 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dcc56c-69f1-4b05-bc26-ed66a36880e1 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:22:35.473193 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dcc56c-69f1-4b05-bc26-ed66a36880e1 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:22:35.474470 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 705/1410] 10.222.0.58 () {54 vars in 918 bytes} [Tue May 17 19:22:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:22:35.480543 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b531d2-0f70-49ca-b2f3-2475945b3311 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:22:35.480997 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b531d2-0f70-49ca-b2f3-2475945b3311 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:22:35.481124 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b531d2-0f70-49ca-b2f3-2475945b3311 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:22:35.498280 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-66b531d2-0f70-49ca-b2f3-2475945b3311 None None] MFA Rules not processed for user `06538aab5f194af7b7c677d6b94aef2a`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:22:35.499476 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-66b531d2-0f70-49ca-b2f3-2475945b3311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:22:35.508283 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 706/1411] 10.222.0.58 () {58 vars in 1015 bytes} [Tue May 17 19:22:35 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 17 19:22:35.522074 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3465e793-ae7e-41d4-a430-e5c31dcec4e6 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:22:35.523037 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3465e793-ae7e-41d4-a430-e5c31dcec4e6 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:22:35.523309 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3465e793-ae7e-41d4-a430-e5c31dcec4e6 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:22:35.524703 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 706/1412] 10.222.0.58 () {54 vars in 918 bytes} [Tue May 17 19:22:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:22:35.529360 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77973cb5-6d06-4d3f-86a3-5d568a5a51aa None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:22:35.529639 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77973cb5-6d06-4d3f-86a3-5d568a5a51aa None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:22:35.529907 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77973cb5-6d06-4d3f-86a3-5d568a5a51aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:22:35.546901 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-77973cb5-6d06-4d3f-86a3-5d568a5a51aa None None] MFA Rules not processed for user `65c1f878b0dc4d63a4f5dcb12fac1dc1`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:22:35.548092 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-77973cb5-6d06-4d3f-86a3-5d568a5a51aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:22:35.556897 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 707/1413] 10.222.0.58 () {58 vars in 1015 bytes} [Tue May 17 19:22:35 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 17 19:22:38.201964 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c08c0c9-c117-43ef-8362-2715c6c73442 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:22:38.202514 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c08c0c9-c117-43ef-8362-2715c6c73442 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:22:38.202514 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c08c0c9-c117-43ef-8362-2715c6c73442 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:22:38.203818 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 707/1414] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 17 19:22:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:22:38.209136 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e444d24-6c4d-4752-ad14-5133bacdd15e None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:22:38.210076 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e444d24-6c4d-4752-ad14-5133bacdd15e None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:22:38.210235 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e444d24-6c4d-4752-ad14-5133bacdd15e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:22:38.228131 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-1e444d24-6c4d-4752-ad14-5133bacdd15e None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:22:38.229314 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-1e444d24-6c4d-4752-ad14-5133bacdd15e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:22:38.236994 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 708/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 17 19:22:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:22:38.276871 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c08c0c9-c117-43ef-8362-2715c6c73442 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:22:38.291402 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c08c0c9-c117-43ef-8362-2715c6c73442 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:22:38.297000 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaea37d-1573-415a-9f39-de307f60c7ff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c1f878b0dc4d63a4f5dcb12fac1dc1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:22:38.297633 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaea37d-1573-415a-9f39-de307f60c7ff service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:22:38.297912 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaea37d-1573-415a-9f39-de307f60c7ff service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:22:38.298193 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaea37d-1573-415a-9f39-de307f60c7ff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:22:38.306247 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aaea37d-1573-415a-9f39-de307f60c7ff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:22:38.307865 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aaea37d-1573-415a-9f39-de307f60c7ff service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:22:38.317013 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 708/1416] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:22:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:22:39.290096 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a926cb-cba2-4999-8b96-6b2bfab41f08 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:22:39.290613 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a926cb-cba2-4999-8b96-6b2bfab41f08 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:22:39.290613 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a926cb-cba2-4999-8b96-6b2bfab41f08 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:22:39.291963 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 709/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 17 19:22:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:22:39.296710 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10ba324-f423-4d64-b0b8-bb4e664568f2 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:22:39.297152 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10ba324-f423-4d64-b0b8-bb4e664568f2 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:22:39.297250 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10ba324-f423-4d64-b0b8-bb4e664568f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:22:39.315011 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-d10ba324-f423-4d64-b0b8-bb4e664568f2 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:22:39.316213 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d10ba324-f423-4d64-b0b8-bb4e664568f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:22:39.323760 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 709/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 17 19:22:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:22:39.497275 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a926cb-cba2-4999-8b96-6b2bfab41f08 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:22:39.511032 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a926cb-cba2-4999-8b96-6b2bfab41f08 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:22:39.516317 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d9ae1a-4e48-469d-ba36-d84047bdbbe7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c1f878b0dc4d63a4f5dcb12fac1dc1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:22:39.516931 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d9ae1a-4e48-469d-ba36-d84047bdbbe7 service nova] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:22:39.517204 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d9ae1a-4e48-469d-ba36-d84047bdbbe7 service nova] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:22:39.517472 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d9ae1a-4e48-469d-ba36-d84047bdbbe7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:22:39.525347 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d9ae1a-4e48-469d-ba36-d84047bdbbe7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:22:39.526913 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d9ae1a-4e48-469d-ba36-d84047bdbbe7 service nova] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:22:39.535828 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 710/1419] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:22:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:22:40.473528 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a426d29-c09d-4aed-bc8f-7fe75e0ef5d0 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:22:40.474095 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a426d29-c09d-4aed-bc8f-7fe75e0ef5d0 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:22:40.474095 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a426d29-c09d-4aed-bc8f-7fe75e0ef5d0 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:22:40.476341 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 710/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 17 19:22:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:22:40.481184 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec88159-4c04-4186-af9f-6dc768a13dd1 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:22:40.481616 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec88159-4c04-4186-af9f-6dc768a13dd1 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:22:40.481731 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec88159-4c04-4186-af9f-6dc768a13dd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:22:40.499144 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-8ec88159-4c04-4186-af9f-6dc768a13dd1 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:22:40.500361 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-8ec88159-4c04-4186-af9f-6dc768a13dd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:22:40.508108 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 711/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 17 19:22:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:22:40.655480 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a426d29-c09d-4aed-bc8f-7fe75e0ef5d0 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:22:40.670361 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a426d29-c09d-4aed-bc8f-7fe75e0ef5d0 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:22:40.675755 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac29996-be89-4fec-9d50-f94d6a3975f0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c1f878b0dc4d63a4f5dcb12fac1dc1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:22:40.676384 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac29996-be89-4fec-9d50-f94d6a3975f0 service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:22:40.676663 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac29996-be89-4fec-9d50-f94d6a3975f0 service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:22:40.676934 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac29996-be89-4fec-9d50-f94d6a3975f0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:22:40.684892 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac29996-be89-4fec-9d50-f94d6a3975f0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:22:40.686473 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac29996-be89-4fec-9d50-f94d6a3975f0 service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:22:40.695691 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 711/1422] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:22:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:22:41.610923 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84a579a-086b-49ec-a247-17316a163ec3 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:22:41.611474 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84a579a-086b-49ec-a247-17316a163ec3 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:22:41.611474 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84a579a-086b-49ec-a247-17316a163ec3 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:22:41.612924 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 712/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 17 19:22:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:22:41.618371 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd34c0b0-539f-4961-bbd2-267d6312c9ed None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:22:41.618804 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd34c0b0-539f-4961-bbd2-267d6312c9ed None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:22:41.618899 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd34c0b0-539f-4961-bbd2-267d6312c9ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:22:41.636154 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-dd34c0b0-539f-4961-bbd2-267d6312c9ed None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:22:41.637348 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-dd34c0b0-539f-4961-bbd2-267d6312c9ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:22:41.645018 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 712/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 17 19:22:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:22:41.792567 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84a579a-086b-49ec-a247-17316a163ec3 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:22:41.806505 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84a579a-086b-49ec-a247-17316a163ec3 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:22:41.811841 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c88e35-265f-432c-ba5a-2831d93fb4ec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c1f878b0dc4d63a4f5dcb12fac1dc1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:22:41.812464 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c88e35-265f-432c-ba5a-2831d93fb4ec service nova] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:22:41.812742 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c88e35-265f-432c-ba5a-2831d93fb4ec service nova] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:22:41.813012 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c88e35-265f-432c-ba5a-2831d93fb4ec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:22:41.821283 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c88e35-265f-432c-ba5a-2831d93fb4ec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:22:41.822843 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c88e35-265f-432c-ba5a-2831d93fb4ec service nova] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:22:41.831880 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 713/1425] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:22:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:22:42.742802 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a9adac-d251-4076-86dc-9b6b382c369c None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:22:42.743415 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a9adac-d251-4076-86dc-9b6b382c369c None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:22:42.743415 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a9adac-d251-4076-86dc-9b6b382c369c None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:22:42.744743 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 713/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 17 19:22:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 17 19:22:42.749439 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a41155-a3a6-4fab-8986-cdb1f2ec4b66 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:22:42.749845 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a41155-a3a6-4fab-8986-cdb1f2ec4b66 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:22:42.749988 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a41155-a3a6-4fab-8986-cdb1f2ec4b66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:22:42.767794 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-f0a41155-a3a6-4fab-8986-cdb1f2ec4b66 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:22:42.768972 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-f0a41155-a3a6-4fab-8986-cdb1f2ec4b66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:22:42.776562 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 714/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 17 19:22:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:22:42.924698 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a9adac-d251-4076-86dc-9b6b382c369c None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:22:42.939453 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a9adac-d251-4076-86dc-9b6b382c369c None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:22:42.945494 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d06399e-e7f9-45ba-bbc7-42c8404ef7a5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c1f878b0dc4d63a4f5dcb12fac1dc1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:22:42.946172 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d06399e-e7f9-45ba-bbc7-42c8404ef7a5 service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:22:42.946473 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d06399e-e7f9-45ba-bbc7-42c8404ef7a5 service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:22:42.946767 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d06399e-e7f9-45ba-bbc7-42c8404ef7a5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:22:42.955219 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d06399e-e7f9-45ba-bbc7-42c8404ef7a5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:22:42.956843 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d06399e-e7f9-45ba-bbc7-42c8404ef7a5 service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:22:42.966157 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 714/1428] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:22:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:22:43.879604 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93edb92-7914-4fe9-be52-ee123662090f None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:22:43.880146 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93edb92-7914-4fe9-be52-ee123662090f None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:22:43.880146 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93edb92-7914-4fe9-be52-ee123662090f None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:22:43.882419 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 715/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 17 19:22:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:22:43.887741 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405eeeb5-ccd9-49cd-abfc-28f7754acfb3 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:22:43.888169 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405eeeb5-ccd9-49cd-abfc-28f7754acfb3 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:22:43.888302 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405eeeb5-ccd9-49cd-abfc-28f7754acfb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:22:43.906594 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-405eeeb5-ccd9-49cd-abfc-28f7754acfb3 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:22:43.907793 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-405eeeb5-ccd9-49cd-abfc-28f7754acfb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:22:43.915381 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 715/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 17 19:22:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:22:44.064460 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93edb92-7914-4fe9-be52-ee123662090f None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:22:44.078279 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93edb92-7914-4fe9-be52-ee123662090f None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:22:44.083537 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bebc01-e11c-4f26-a652-bb12e7872d47 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c1f878b0dc4d63a4f5dcb12fac1dc1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:22:44.084161 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bebc01-e11c-4f26-a652-bb12e7872d47 service nova] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:22:44.084435 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bebc01-e11c-4f26-a652-bb12e7872d47 service nova] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:22:44.084706 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bebc01-e11c-4f26-a652-bb12e7872d47 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:22:44.092428 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bebc01-e11c-4f26-a652-bb12e7872d47 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:22:44.093987 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bebc01-e11c-4f26-a652-bb12e7872d47 service nova] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:22:44.103760 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 716/1431] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:22:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:22:45.054452 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad5423c-4b0b-474d-856e-cbe8fa0501b3 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:22:45.054983 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad5423c-4b0b-474d-856e-cbe8fa0501b3 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:22:45.054983 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad5423c-4b0b-474d-856e-cbe8fa0501b3 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:22:45.056416 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 716/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 17 19:22:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:22:45.061390 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e4f1a9-517c-454f-98db-04d2e87d1fbd None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:22:45.061816 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e4f1a9-517c-454f-98db-04d2e87d1fbd None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:22:45.061922 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e4f1a9-517c-454f-98db-04d2e87d1fbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:22:45.079302 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-02e4f1a9-517c-454f-98db-04d2e87d1fbd None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:22:45.080506 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-02e4f1a9-517c-454f-98db-04d2e87d1fbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:22:45.088578 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 717/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 17 19:22:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:22:45.234048 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad5423c-4b0b-474d-856e-cbe8fa0501b3 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:22:45.248092 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad5423c-4b0b-474d-856e-cbe8fa0501b3 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:22:45.253398 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe0f082-9909-4b81-897b-83858a52fb3b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c1f878b0dc4d63a4f5dcb12fac1dc1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:22:45.254026 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe0f082-9909-4b81-897b-83858a52fb3b service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:22:45.254305 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe0f082-9909-4b81-897b-83858a52fb3b service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:22:45.254581 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe0f082-9909-4b81-897b-83858a52fb3b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:22:45.262826 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe0f082-9909-4b81-897b-83858a52fb3b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:22:45.264432 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe0f082-9909-4b81-897b-83858a52fb3b service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:22:45.274150 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 717/1434] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:22:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:22:46.255274 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378b8156-7753-4124-9a6d-f22e69d6c57a None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:22:46.255830 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378b8156-7753-4124-9a6d-f22e69d6c57a None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:22:46.255830 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378b8156-7753-4124-9a6d-f22e69d6c57a None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:22:46.257632 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 718/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 17 19:22:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:22:46.262526 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fbfe0f-11b2-40ed-9a00-a6b82924ee2e None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:22:46.262917 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fbfe0f-11b2-40ed-9a00-a6b82924ee2e None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:22:46.263098 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fbfe0f-11b2-40ed-9a00-a6b82924ee2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:22:46.281266 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-74fbfe0f-11b2-40ed-9a00-a6b82924ee2e None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:22:46.282454 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-74fbfe0f-11b2-40ed-9a00-a6b82924ee2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:22:46.290273 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 718/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 17 19:22:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:22:46.437388 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378b8156-7753-4124-9a6d-f22e69d6c57a None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:22:46.451738 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378b8156-7753-4124-9a6d-f22e69d6c57a None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:22:46.457323 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc33748-4247-48c4-8c0e-34396808f9e0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c1f878b0dc4d63a4f5dcb12fac1dc1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:22:46.457954 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc33748-4247-48c4-8c0e-34396808f9e0 service nova] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:22:46.458235 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc33748-4247-48c4-8c0e-34396808f9e0 service nova] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:22:46.458532 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc33748-4247-48c4-8c0e-34396808f9e0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:22:46.466535 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc33748-4247-48c4-8c0e-34396808f9e0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:22:46.468153 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc33748-4247-48c4-8c0e-34396808f9e0 service nova] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:22:46.477591 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 719/1437] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:22:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:22:47.478097 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c8347a-85f9-4d04-8414-449136e755fc None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:22:47.478631 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c8347a-85f9-4d04-8414-449136e755fc None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:22:47.478631 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c8347a-85f9-4d04-8414-449136e755fc None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:22:47.480034 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 719/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 17 19:22:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:22:47.484938 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd3b3e5-3807-48cb-a301-ab54c4653d88 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:22:47.485370 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd3b3e5-3807-48cb-a301-ab54c4653d88 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:22:47.485490 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd3b3e5-3807-48cb-a301-ab54c4653d88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:22:47.503744 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-ebd3b3e5-3807-48cb-a301-ab54c4653d88 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:22:47.504861 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-ebd3b3e5-3807-48cb-a301-ab54c4653d88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:22:47.513334 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 720/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 17 19:22:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:22:47.660262 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c8347a-85f9-4d04-8414-449136e755fc None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:22:47.674676 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c8347a-85f9-4d04-8414-449136e755fc None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:22:47.680624 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2d02b1-517b-4bca-9fb8-1f291ec3de1b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c1f878b0dc4d63a4f5dcb12fac1dc1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:22:47.681245 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2d02b1-517b-4bca-9fb8-1f291ec3de1b service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:22:47.681523 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2d02b1-517b-4bca-9fb8-1f291ec3de1b service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:22:47.681793 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2d02b1-517b-4bca-9fb8-1f291ec3de1b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:22:47.689542 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2d02b1-517b-4bca-9fb8-1f291ec3de1b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:22:47.691128 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2d02b1-517b-4bca-9fb8-1f291ec3de1b service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:22:47.700302 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 720/1440] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:22:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:22:49.759481 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7bc629-5835-4706-9cd5-817ba21a4438 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:22:49.760090 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7bc629-5835-4706-9cd5-817ba21a4438 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:22:49.760090 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7bc629-5835-4706-9cd5-817ba21a4438 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:22:49.761426 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 721/1441] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 17 19:22:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:22:49.766785 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17b80f9-cfe9-4b27-9144-728df94d913b None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:22:49.767225 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17b80f9-cfe9-4b27-9144-728df94d913b None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:22:49.767337 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17b80f9-cfe9-4b27-9144-728df94d913b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:22:49.785470 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-f17b80f9-cfe9-4b27-9144-728df94d913b None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:22:49.786633 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-f17b80f9-cfe9-4b27-9144-728df94d913b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:22:49.794241 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 721/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 17 19:22:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:22:49.831923 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7bc629-5835-4706-9cd5-817ba21a4438 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:22:49.842792 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7bc629-5835-4706-9cd5-817ba21a4438 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:22:49.846275 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff4ef95-fa2a-4373-9717-72887bfef958 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:22:49.846894 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff4ef95-fa2a-4373-9717-72887bfef958 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:22:49.847172 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff4ef95-fa2a-4373-9717-72887bfef958 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:22:49.847447 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff4ef95-fa2a-4373-9717-72887bfef958 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:22:49.856327 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff4ef95-fa2a-4373-9717-72887bfef958 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:22:49.857788 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff4ef95-fa2a-4373-9717-72887bfef958 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:22:49.867028 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 722/1443] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:22:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:22:52.218350 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11ecb95-c43b-4db4-87bc-867441d642e6 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:22:52.218878 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11ecb95-c43b-4db4-87bc-867441d642e6 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:22:52.218878 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11ecb95-c43b-4db4-87bc-867441d642e6 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:22:52.220276 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 722/1444] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 17 19:22:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:22:52.226253 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84c8563-180a-4ae2-af13-c3340bc262b3 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:22:52.226680 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84c8563-180a-4ae2-af13-c3340bc262b3 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:22:52.226801 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84c8563-180a-4ae2-af13-c3340bc262b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:22:52.245302 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-f84c8563-180a-4ae2-af13-c3340bc262b3 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:22:52.246490 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-f84c8563-180a-4ae2-af13-c3340bc262b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:22:52.254201 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 723/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 17 19:22:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 17 19:22:52.290856 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11ecb95-c43b-4db4-87bc-867441d642e6 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:22:52.301327 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11ecb95-c43b-4db4-87bc-867441d642e6 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:22:52.305451 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad71680-990a-430b-b309-4fc0670548cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:22:52.306078 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad71680-990a-430b-b309-4fc0670548cb service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:22:52.306353 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad71680-990a-430b-b309-4fc0670548cb service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:22:52.306624 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad71680-990a-430b-b309-4fc0670548cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:22:52.315182 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad71680-990a-430b-b309-4fc0670548cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:22:52.316657 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad71680-990a-430b-b309-4fc0670548cb service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:22:52.325722 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 723/1446] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:22:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:22:54.896902 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186efcb5-84aa-4671-9a53-983d2672c8f4 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:22:54.897563 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186efcb5-84aa-4671-9a53-983d2672c8f4 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:22:54.897563 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186efcb5-84aa-4671-9a53-983d2672c8f4 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:22:54.898839 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 724/1447] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 17 19:22:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:22:54.904223 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478dc286-bc2c-4801-9e46-da73a6201daa None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:22:54.904650 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478dc286-bc2c-4801-9e46-da73a6201daa None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:22:54.904778 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478dc286-bc2c-4801-9e46-da73a6201daa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:22:54.922967 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-478dc286-bc2c-4801-9e46-da73a6201daa None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:22:54.924207 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-478dc286-bc2c-4801-9e46-da73a6201daa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:22:54.932476 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 724/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 17 19:22:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:22:54.970177 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186efcb5-84aa-4671-9a53-983d2672c8f4 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:22:54.971864 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-186efcb5-84aa-4671-9a53-983d2672c8f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:22:54.984795 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186efcb5-84aa-4671-9a53-983d2672c8f4 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:22:54.988313 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3ed67c-a573-43a0-9e9c-e2b739e2ff40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:22:54.988939 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3ed67c-a573-43a0-9e9c-e2b739e2ff40 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:22:54.989219 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3ed67c-a573-43a0-9e9c-e2b739e2ff40 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:22:54.989491 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3ed67c-a573-43a0-9e9c-e2b739e2ff40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:22:54.998549 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3ed67c-a573-43a0-9e9c-e2b739e2ff40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:22:55.000002 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3ed67c-a573-43a0-9e9c-e2b739e2ff40 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:22:55.009683 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 725/1449] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:22:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:22:57.309364 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875400e0-117a-496c-b965-705e1706c966 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:22:57.309892 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875400e0-117a-496c-b965-705e1706c966 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:22:57.309892 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875400e0-117a-496c-b965-705e1706c966 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:22:57.311195 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 725/1450] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 17 19:22:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:22:57.316559 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a550f03a-3a9d-402a-aa46-82138e37cb0a None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:22:57.316964 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a550f03a-3a9d-402a-aa46-82138e37cb0a None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:22:57.317141 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a550f03a-3a9d-402a-aa46-82138e37cb0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:22:57.334220 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-a550f03a-3a9d-402a-aa46-82138e37cb0a None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:22:57.335404 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-a550f03a-3a9d-402a-aa46-82138e37cb0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:22:57.343033 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 726/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 17 19:22:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:22:57.380877 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875400e0-117a-496c-b965-705e1706c966 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:22:57.391905 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875400e0-117a-496c-b965-705e1706c966 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:22:57.395979 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58146ead-ae25-4aa7-8c63-628b3ea34cd8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:22:57.396874 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58146ead-ae25-4aa7-8c63-628b3ea34cd8 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:22:57.397236 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58146ead-ae25-4aa7-8c63-628b3ea34cd8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:22:57.397513 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58146ead-ae25-4aa7-8c63-628b3ea34cd8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:22:57.407034 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58146ead-ae25-4aa7-8c63-628b3ea34cd8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:22:57.408495 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58146ead-ae25-4aa7-8c63-628b3ea34cd8 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:22:57.418682 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 726/1452] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:22:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:22:59.760006 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adc34c5-2de4-4543-85ea-fd41271d93b5 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:22:59.760586 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adc34c5-2de4-4543-85ea-fd41271d93b5 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:22:59.760586 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adc34c5-2de4-4543-85ea-fd41271d93b5 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:22:59.761936 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 727/1453] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 17 19:22:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:22:59.767239 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac45c345-6181-407f-bd97-239b385d1176 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:22:59.767680 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac45c345-6181-407f-bd97-239b385d1176 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:22:59.767826 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac45c345-6181-407f-bd97-239b385d1176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:22:59.785421 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-ac45c345-6181-407f-bd97-239b385d1176 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:22:59.786590 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-ac45c345-6181-407f-bd97-239b385d1176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:22:59.794275 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 727/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 17 19:22:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:22:59.830997 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adc34c5-2de4-4543-85ea-fd41271d93b5 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:22:59.841775 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adc34c5-2de4-4543-85ea-fd41271d93b5 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:22:59.845236 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faec1bf-3bb3-4e8f-bddd-13d8f48f42f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:22:59.845853 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faec1bf-3bb3-4e8f-bddd-13d8f48f42f4 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:22:59.846129 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faec1bf-3bb3-4e8f-bddd-13d8f48f42f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:22:59.846397 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faec1bf-3bb3-4e8f-bddd-13d8f48f42f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:22:59.855170 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faec1bf-3bb3-4e8f-bddd-13d8f48f42f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:22:59.856612 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faec1bf-3bb3-4e8f-bddd-13d8f48f42f4 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:22:59.866402 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 728/1455] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:22:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:23:02.113915 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e33bdf6-bc5c-4d8c-8ba2-eb9deaa7965e None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:23:02.114434 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e33bdf6-bc5c-4d8c-8ba2-eb9deaa7965e None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:23:02.114434 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e33bdf6-bc5c-4d8c-8ba2-eb9deaa7965e None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:23:02.116540 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 728/1456] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 17 19:23:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:23:02.121888 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89ffb83-ec75-47e2-a8b5-4b914625eaa4 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:23:02.122313 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89ffb83-ec75-47e2-a8b5-4b914625eaa4 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:23:02.122460 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89ffb83-ec75-47e2-a8b5-4b914625eaa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:23:02.139503 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-d89ffb83-ec75-47e2-a8b5-4b914625eaa4 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:23:02.140681 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-d89ffb83-ec75-47e2-a8b5-4b914625eaa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:23:02.148821 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 729/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 17 19:23:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:23:02.186390 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e33bdf6-bc5c-4d8c-8ba2-eb9deaa7965e None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:23:02.196825 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e33bdf6-bc5c-4d8c-8ba2-eb9deaa7965e None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:23:02.200709 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782ea9bb-b0f8-4a5f-892f-b7aa5e9fb10e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:23:02.201300 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782ea9bb-b0f8-4a5f-892f-b7aa5e9fb10e service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:23:02.201575 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782ea9bb-b0f8-4a5f-892f-b7aa5e9fb10e service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:23:02.201846 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782ea9bb-b0f8-4a5f-892f-b7aa5e9fb10e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:23:02.210852 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782ea9bb-b0f8-4a5f-892f-b7aa5e9fb10e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:23:02.212313 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782ea9bb-b0f8-4a5f-892f-b7aa5e9fb10e service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:23:02.221336 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 729/1458] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:23:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:23:04.451741 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129f5afd-97b9-4ebe-a2b9-4e8002ade3bc None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:23:04.452320 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129f5afd-97b9-4ebe-a2b9-4e8002ade3bc None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:23:04.452320 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129f5afd-97b9-4ebe-a2b9-4e8002ade3bc None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:23:04.453666 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 730/1459] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 17 19:23:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:23:04.459327 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f545711-000b-4d44-b27a-de4b1fa7c6d8 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:23:04.459752 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f545711-000b-4d44-b27a-de4b1fa7c6d8 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:23:04.459919 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f545711-000b-4d44-b27a-de4b1fa7c6d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:23:04.477979 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-2f545711-000b-4d44-b27a-de4b1fa7c6d8 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:23:04.479157 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-2f545711-000b-4d44-b27a-de4b1fa7c6d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:23:04.486804 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 730/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 17 19:23:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:23:04.523145 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129f5afd-97b9-4ebe-a2b9-4e8002ade3bc None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:23:04.533555 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129f5afd-97b9-4ebe-a2b9-4e8002ade3bc None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:23:04.538567 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e0d279-5c98-4360-8891-e5529dae16b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:23:04.538567 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e0d279-5c98-4360-8891-e5529dae16b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:23:04.538567 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e0d279-5c98-4360-8891-e5529dae16b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:23:04.538905 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e0d279-5c98-4360-8891-e5529dae16b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:23:04.547362 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e0d279-5c98-4360-8891-e5529dae16b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:23:04.548792 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e0d279-5c98-4360-8891-e5529dae16b0 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:23:04.557822 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 731/1461] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:23:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:23:06.799324 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47c4e60-d460-4eb0-a6f6-4b75910fa673 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:23:06.799957 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47c4e60-d460-4eb0-a6f6-4b75910fa673 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:23:06.799957 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47c4e60-d460-4eb0-a6f6-4b75910fa673 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:23:06.801265 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 731/1462] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 17 19:23:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:23:06.806578 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fedff02-ca96-4886-97b4-c1808d7e501e None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:23:06.807041 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fedff02-ca96-4886-97b4-c1808d7e501e None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:23:06.807165 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fedff02-ca96-4886-97b4-c1808d7e501e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:23:06.825086 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-7fedff02-ca96-4886-97b4-c1808d7e501e None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:23:06.826272 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-7fedff02-ca96-4886-97b4-c1808d7e501e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:23:06.834575 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 732/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 17 19:23:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:23:06.871346 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47c4e60-d460-4eb0-a6f6-4b75910fa673 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:23:06.881930 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47c4e60-d460-4eb0-a6f6-4b75910fa673 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:23:06.885449 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b56a17-810f-4b8e-b947-f5b37047d4bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:23:06.886074 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b56a17-810f-4b8e-b947-f5b37047d4bb service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:23:06.886354 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b56a17-810f-4b8e-b947-f5b37047d4bb service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:23:06.886624 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b56a17-810f-4b8e-b947-f5b37047d4bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:23:06.895403 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b56a17-810f-4b8e-b947-f5b37047d4bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:23:06.897376 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b56a17-810f-4b8e-b947-f5b37047d4bb service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:23:06.906668 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 732/1464] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:23:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:23:09.097880 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba0bcb2-a267-44fd-993e-3bc939b9dd9b None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:23:09.098413 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba0bcb2-a267-44fd-993e-3bc939b9dd9b None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:23:09.098413 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba0bcb2-a267-44fd-993e-3bc939b9dd9b None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:23:09.100844 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 733/1465] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 17 19:23:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:23:09.106102 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771e1cfa-767b-4f2e-9109-70f1e7a9ae34 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:23:09.106493 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771e1cfa-767b-4f2e-9109-70f1e7a9ae34 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:23:09.106639 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771e1cfa-767b-4f2e-9109-70f1e7a9ae34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:23:09.123673 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-771e1cfa-767b-4f2e-9109-70f1e7a9ae34 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:23:09.124862 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-771e1cfa-767b-4f2e-9109-70f1e7a9ae34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:23:09.132443 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 733/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 17 19:23:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 17 19:23:09.167912 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba0bcb2-a267-44fd-993e-3bc939b9dd9b None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:23:09.178175 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba0bcb2-a267-44fd-993e-3bc939b9dd9b None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:23:09.181627 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6842001c-2e9f-4a1a-8557-04ec43c6445b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:23:09.182242 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6842001c-2e9f-4a1a-8557-04ec43c6445b service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:23:09.182516 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6842001c-2e9f-4a1a-8557-04ec43c6445b service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:23:09.182788 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6842001c-2e9f-4a1a-8557-04ec43c6445b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:23:09.191255 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6842001c-2e9f-4a1a-8557-04ec43c6445b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:23:09.192699 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6842001c-2e9f-4a1a-8557-04ec43c6445b service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:23:09.201704 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 734/1467] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:23:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:23:11.390593 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e466a14-7126-4749-9d97-e969ea7a5684 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:23:11.391151 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e466a14-7126-4749-9d97-e969ea7a5684 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:23:11.391151 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e466a14-7126-4749-9d97-e969ea7a5684 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:23:11.392441 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 734/1468] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 17 19:23:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:23:11.398233 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac363ff-3ab4-4464-988d-f70c1cf92c84 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:23:11.398655 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac363ff-3ab4-4464-988d-f70c1cf92c84 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:23:11.398773 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac363ff-3ab4-4464-988d-f70c1cf92c84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:23:11.416592 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-9ac363ff-3ab4-4464-988d-f70c1cf92c84 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:23:11.417779 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-9ac363ff-3ab4-4464-988d-f70c1cf92c84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:23:11.425490 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 735/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 17 19:23:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:23:11.460949 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e466a14-7126-4749-9d97-e969ea7a5684 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:23:11.471246 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e466a14-7126-4749-9d97-e969ea7a5684 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:23:11.475640 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05bcd23-8e83-495a-aa29-31087cb2529f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:23:11.476259 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05bcd23-8e83-495a-aa29-31087cb2529f service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:23:11.476532 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05bcd23-8e83-495a-aa29-31087cb2529f service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:23:11.476801 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05bcd23-8e83-495a-aa29-31087cb2529f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:23:11.485324 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05bcd23-8e83-495a-aa29-31087cb2529f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:23:11.486739 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05bcd23-8e83-495a-aa29-31087cb2529f service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:23:11.495775 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 735/1470] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:23:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:23:44.607840 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441a390c-72f3-4852-86dc-7582c5b68c96 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:23:44.608460 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441a390c-72f3-4852-86dc-7582c5b68c96 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:23:44.608460 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441a390c-72f3-4852-86dc-7582c5b68c96 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:23:44.609756 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 736/1471] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:23:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:23:44.614807 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d0f608-1b23-402e-9b17-0481f5f3937a None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:23:44.615249 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d0f608-1b23-402e-9b17-0481f5f3937a None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:23:44.615391 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d0f608-1b23-402e-9b17-0481f5f3937a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:23:44.633313 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-f3d0f608-1b23-402e-9b17-0481f5f3937a None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:23:44.634504 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-f3d0f608-1b23-402e-9b17-0481f5f3937a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:23:44.642240 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 736/1472] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:23:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:23:44.688223 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4745a7-cfce-4726-849f-fd890e639c38 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:23:44.688505 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4745a7-cfce-4726-849f-fd890e639c38 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:23:44.688775 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4745a7-cfce-4726-849f-fd890e639c38 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:23:44.690120 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 737/1473] 10.222.0.58 () {54 vars in 962 bytes} [Tue May 17 19:23:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:23:44.695286 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d005a1-006c-4091-9efc-4e5aff76951a None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:23:44.695618 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d005a1-006c-4091-9efc-4e5aff76951a None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:23:44.695903 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d005a1-006c-4091-9efc-4e5aff76951a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:23:44.713925 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-66d005a1-006c-4091-9efc-4e5aff76951a None None] MFA Rules not processed for user `57e888884d2b4629bfd12d1034ce5be4`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:23:44.715105 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-66d005a1-006c-4091-9efc-4e5aff76951a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:23:44.721532 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 737/1474] 10.222.0.58 () {58 vars in 1059 bytes} [Tue May 17 19:23:44 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:23:44.728697 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4745a7-cfce-4726-849f-fd890e639c38 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:23:44.738820 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4745a7-cfce-4726-849f-fd890e639c38 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:23:44.742251 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bf365b-2b55-4b4b-a6c4-5d66bb2e29f8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57e888884d2b4629bfd12d1034ce5be4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:23:44.742851 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bf365b-2b55-4b4b-a6c4-5d66bb2e29f8 service glance] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:23:44.743124 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bf365b-2b55-4b4b-a6c4-5d66bb2e29f8 service glance] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:23:44.743392 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bf365b-2b55-4b4b-a6c4-5d66bb2e29f8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:23:44.751871 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bf365b-2b55-4b4b-a6c4-5d66bb2e29f8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:23:44.753856 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bf365b-2b55-4b4b-a6c4-5d66bb2e29f8 service glance] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:23:44.763053 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 738/1475] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:23:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 17 19:23:47.048702 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173d5526-3d70-4cba-813b-e180369f05b6 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:23:47.049257 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173d5526-3d70-4cba-813b-e180369f05b6 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:23:47.049257 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173d5526-3d70-4cba-813b-e180369f05b6 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:23:47.051232 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 738/1476] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:23:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:23:47.056521 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a010e219-8397-4328-8591-37360350b477 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:23:47.056938 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a010e219-8397-4328-8591-37360350b477 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:23:47.057064 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a010e219-8397-4328-8591-37360350b477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:23:47.074062 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-a010e219-8397-4328-8591-37360350b477 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:23:47.075230 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-a010e219-8397-4328-8591-37360350b477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:23:47.082804 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 739/1477] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:23:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 17 19:23:47.105923 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173d5526-3d70-4cba-813b-e180369f05b6 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:23:47.118738 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173d5526-3d70-4cba-813b-e180369f05b6 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:23:47.123176 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ab6909-6cc8-4267-8309-8b1ce68c1a32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:23:47.126609 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ab6909-6cc8-4267-8309-8b1ce68c1a32 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:23:47.126609 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ab6909-6cc8-4267-8309-8b1ce68c1a32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:23:47.126609 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ab6909-6cc8-4267-8309-8b1ce68c1a32 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:23:47.126609 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ab6909-6cc8-4267-8309-8b1ce68c1a32 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:23:47.126953 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: WARNING py.warnings [None req-b8ab6909-6cc8-4267-8309-8b1ce68c1a32 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 17 19:23:47.126953 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: warnings.warn(msg) May 17 19:23:47.126953 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: May 17 19:23:47.127966 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ab6909-6cc8-4267-8309-8b1ce68c1a32 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:23:47.147058 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 739/1478] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 17 19:23:47 2022] GET /identity/v3/projects => generated 2553 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:23:49.358486 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890ab38b-7770-429f-a773-d7d16a656767 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:23:49.359085 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890ab38b-7770-429f-a773-d7d16a656767 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:23:49.359085 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890ab38b-7770-429f-a773-d7d16a656767 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:23:49.360350 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 740/1479] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:23:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:23:49.366124 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b56ed78-3879-47df-abb0-ed16548d4947 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:23:49.366551 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b56ed78-3879-47df-abb0-ed16548d4947 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:23:49.366654 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b56ed78-3879-47df-abb0-ed16548d4947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:23:49.383857 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-8b56ed78-3879-47df-abb0-ed16548d4947 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:23:49.385024 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-8b56ed78-3879-47df-abb0-ed16548d4947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:23:49.392573 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 740/1480] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:23:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 17 19:23:49.433264 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890ab38b-7770-429f-a773-d7d16a656767 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:23:49.447268 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890ab38b-7770-429f-a773-d7d16a656767 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:23:49.452658 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cb12c7-86f8-43c3-9d51-2a6c37ca01a6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c1f878b0dc4d63a4f5dcb12fac1dc1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:23:49.453271 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cb12c7-86f8-43c3-9d51-2a6c37ca01a6 service nova] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:23:49.453548 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cb12c7-86f8-43c3-9d51-2a6c37ca01a6 service nova] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:23:49.453824 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cb12c7-86f8-43c3-9d51-2a6c37ca01a6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:23:49.462234 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cb12c7-86f8-43c3-9d51-2a6c37ca01a6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:23:49.463839 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cb12c7-86f8-43c3-9d51-2a6c37ca01a6 service nova] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:23:49.473549 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 741/1481] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:23:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:23:51.860700 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c8d982-a77c-46f9-81bb-c3d847d5ad45 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:23:51.861201 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c8d982-a77c-46f9-81bb-c3d847d5ad45 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:23:51.861201 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c8d982-a77c-46f9-81bb-c3d847d5ad45 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:23:51.863089 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 741/1482] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:23:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:23:51.868487 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f7ee73-28ed-42e7-ba5d-c2f9bb7321da None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:23:51.868949 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f7ee73-28ed-42e7-ba5d-c2f9bb7321da None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:23:51.869058 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f7ee73-28ed-42e7-ba5d-c2f9bb7321da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:23:51.886759 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-18f7ee73-28ed-42e7-ba5d-c2f9bb7321da None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:23:51.887973 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-18f7ee73-28ed-42e7-ba5d-c2f9bb7321da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:23:51.895605 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 742/1483] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:23:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 17 19:23:51.931511 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c8d982-a77c-46f9-81bb-c3d847d5ad45 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:23:51.942541 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c8d982-a77c-46f9-81bb-c3d847d5ad45 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:23:51.946183 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d32f32e-d163-4075-90e8-d213b5a7bc24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:23:51.946813 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d32f32e-d163-4075-90e8-d213b5a7bc24 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:23:51.947088 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d32f32e-d163-4075-90e8-d213b5a7bc24 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:23:51.947362 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d32f32e-d163-4075-90e8-d213b5a7bc24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:23:51.956397 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d32f32e-d163-4075-90e8-d213b5a7bc24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:23:51.957818 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d32f32e-d163-4075-90e8-d213b5a7bc24 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:23:51.966790 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 742/1484] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:23:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:23:54.207881 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8046e385-0a5d-4afa-8486-71a6e5473476 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:23:54.208395 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8046e385-0a5d-4afa-8486-71a6e5473476 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:23:54.208395 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8046e385-0a5d-4afa-8486-71a6e5473476 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:23:54.209746 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 743/1485] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:23:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:23:54.214979 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520dc4dd-d7a6-46c3-9692-08c7ad44d95a None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:23:54.215401 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520dc4dd-d7a6-46c3-9692-08c7ad44d95a None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:23:54.215564 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520dc4dd-d7a6-46c3-9692-08c7ad44d95a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:23:54.233088 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-520dc4dd-d7a6-46c3-9692-08c7ad44d95a None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:23:54.234277 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-520dc4dd-d7a6-46c3-9692-08c7ad44d95a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:23:54.242531 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 743/1486] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:23:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 17 19:23:54.279650 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8046e385-0a5d-4afa-8486-71a6e5473476 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:23:54.291052 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8046e385-0a5d-4afa-8486-71a6e5473476 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:23:54.294536 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5cd091-5e73-4985-bbb9-bdfe5d1ebb10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:23:54.295150 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5cd091-5e73-4985-bbb9-bdfe5d1ebb10 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:23:54.295423 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5cd091-5e73-4985-bbb9-bdfe5d1ebb10 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:23:54.295720 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5cd091-5e73-4985-bbb9-bdfe5d1ebb10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:23:54.304734 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5cd091-5e73-4985-bbb9-bdfe5d1ebb10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:23:54.306142 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5cd091-5e73-4985-bbb9-bdfe5d1ebb10 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:23:54.315161 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 744/1487] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:23:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:23:56.635518 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b82dfb-d04c-4148-8860-6bff3acf3362 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:23:56.636178 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b82dfb-d04c-4148-8860-6bff3acf3362 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:23:56.636178 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b82dfb-d04c-4148-8860-6bff3acf3362 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:23:56.637480 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 744/1488] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:23:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:23:56.642759 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0830c8e3-166c-4ba8-9616-6244dea00e5a None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:23:56.643192 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0830c8e3-166c-4ba8-9616-6244dea00e5a None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:23:56.643348 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0830c8e3-166c-4ba8-9616-6244dea00e5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:23:56.661237 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-0830c8e3-166c-4ba8-9616-6244dea00e5a None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:23:56.662430 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-0830c8e3-166c-4ba8-9616-6244dea00e5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:23:56.670138 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 745/1489] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:23:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:23:56.709391 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b82dfb-d04c-4148-8860-6bff3acf3362 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:23:56.719801 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b82dfb-d04c-4148-8860-6bff3acf3362 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:23:56.723731 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d5d8b3-f561-4c5b-9476-6cc0fbfd9e02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:23:56.724620 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d5d8b3-f561-4c5b-9476-6cc0fbfd9e02 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:23:56.724895 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d5d8b3-f561-4c5b-9476-6cc0fbfd9e02 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:23:56.725165 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d5d8b3-f561-4c5b-9476-6cc0fbfd9e02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:23:56.733661 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d5d8b3-f561-4c5b-9476-6cc0fbfd9e02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:23:56.735088 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d5d8b3-f561-4c5b-9476-6cc0fbfd9e02 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:23:56.744780 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 745/1490] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:23:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:23:59.298426 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e887a2-01ef-48c6-98ea-fb73e35d3f0a None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:23:59.298961 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e887a2-01ef-48c6-98ea-fb73e35d3f0a None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:23:59.298961 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e887a2-01ef-48c6-98ea-fb73e35d3f0a None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:23:59.300850 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 746/1491] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 17 19:23:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:23:59.306156 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67a0513-5fc4-41c6-b059-38a35cb585ec None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:23:59.306598 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67a0513-5fc4-41c6-b059-38a35cb585ec None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:23:59.306704 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67a0513-5fc4-41c6-b059-38a35cb585ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:23:59.324553 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-e67a0513-5fc4-41c6-b059-38a35cb585ec None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:23:59.325721 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-e67a0513-5fc4-41c6-b059-38a35cb585ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:23:59.333313 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 746/1492] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 17 19:23:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:23:59.371123 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e887a2-01ef-48c6-98ea-fb73e35d3f0a None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:23:59.382239 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e887a2-01ef-48c6-98ea-fb73e35d3f0a None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:23:59.385791 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0c8646-7824-4f13-a625-479ed64a1d16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:23:59.386412 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0c8646-7824-4f13-a625-479ed64a1d16 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:23:59.386690 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0c8646-7824-4f13-a625-479ed64a1d16 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:23:59.386962 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0c8646-7824-4f13-a625-479ed64a1d16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:23:59.395640 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0c8646-7824-4f13-a625-479ed64a1d16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:23:59.397084 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0c8646-7824-4f13-a625-479ed64a1d16 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:23:59.406160 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 747/1493] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:23:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:25:56.605803 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfc5cff-228c-4449-aff5-a138a289a0ca None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:25:56.606392 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfc5cff-228c-4449-aff5-a138a289a0ca None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:25:56.606392 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfc5cff-228c-4449-aff5-a138a289a0ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:25:56.624022 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-7cfc5cff-228c-4449-aff5-a138a289a0ca None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:25:56.625212 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-7cfc5cff-228c-4449-aff5-a138a289a0ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:25:56.633303 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 747/1494] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:25:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:25:56.642174 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6b5ddb-cd0a-43d3-a858-eb0bfc09c058 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:25:56.642690 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6b5ddb-cd0a-43d3-a858-eb0bfc09c058 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:25:56.642802 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6b5ddb-cd0a-43d3-a858-eb0bfc09c058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:25:56.660747 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-6f6b5ddb-cd0a-43d3-a858-eb0bfc09c058 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:25:56.661942 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-6f6b5ddb-cd0a-43d3-a858-eb0bfc09c058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:25:56.669733 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 748/1495] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:25:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:25:56.674074 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfc5cff-228c-4449-aff5-a138a289a0ca None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:25:56.683425 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfc5cff-228c-4449-aff5-a138a289a0ca None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:25:56.687893 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc70326-0afb-4fd1-b53a-789befa90c14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:25:56.688528 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc70326-0afb-4fd1-b53a-789befa90c14 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:25:56.688804 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc70326-0afb-4fd1-b53a-789befa90c14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:25:56.689073 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc70326-0afb-4fd1-b53a-789befa90c14 admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:25:56.689778 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc70326-0afb-4fd1-b53a-789befa90c14 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:25:56.690077 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc70326-0afb-4fd1-b53a-789befa90c14 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:25:56.691078 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: WARNING py.warnings [None req-9bc70326-0afb-4fd1-b53a-789befa90c14 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 17 19:25:56.691078 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: warnings.warn(msg) May 17 19:25:56.691078 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: May 17 19:25:56.691998 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc70326-0afb-4fd1-b53a-789befa90c14 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:25:56.703450 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 748/1496] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 17 19:25:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:25:56.711858 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6b5ddb-cd0a-43d3-a858-eb0bfc09c058 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:25:56.720728 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6b5ddb-cd0a-43d3-a858-eb0bfc09c058 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:25:56.725574 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c67c0a-fcdd-4bf2-bfc6-f545e7de58f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:25:56.726205 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c67c0a-fcdd-4bf2-bfc6-f545e7de58f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:25:56.726487 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c67c0a-fcdd-4bf2-bfc6-f545e7de58f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:25:56.726761 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c67c0a-fcdd-4bf2-bfc6-f545e7de58f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:25:56.727551 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c67c0a-fcdd-4bf2-bfc6-f545e7de58f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:25:56.729270 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: WARNING py.warnings [None req-48c67c0a-fcdd-4bf2-bfc6-f545e7de58f7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 17 19:25:56.729270 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: warnings.warn(msg) May 17 19:25:56.729270 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: May 17 19:25:56.730448 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c67c0a-fcdd-4bf2-bfc6-f545e7de58f7 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:25:56.831025 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 749/1497] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 17 19:25:56 2022] POST /identity/v3/projects => generated 379 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 17 19:25:56.835857 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc70326-0afb-4fd1-b53a-789befa90c14 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:25:56.845639 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc70326-0afb-4fd1-b53a-789befa90c14 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:25:56.900286 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f931117a-6f5e-4171-b6b4-e83e0a2f1890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:25:56.900950 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f931117a-6f5e-4171-b6b4-e83e0a2f1890 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:25:56.901230 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f931117a-6f5e-4171-b6b4-e83e0a2f1890 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:25:56.901503 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f931117a-6f5e-4171-b6b4-e83e0a2f1890 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:25:56.902187 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f931117a-6f5e-4171-b6b4-e83e0a2f1890 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:25:56.903218 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: WARNING py.warnings [None req-f931117a-6f5e-4171-b6b4-e83e0a2f1890 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 17 19:25:56.903218 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: warnings.warn(msg) May 17 19:25:56.903218 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: May 17 19:25:56.904466 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f931117a-6f5e-4171-b6b4-e83e0a2f1890 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:25:57.001944 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 749/1498] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 17 19:25:56 2022] POST /identity/v3/users => generated 290 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:25:57.005954 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c67c0a-fcdd-4bf2-bfc6-f545e7de58f7 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:25:57.016045 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c67c0a-fcdd-4bf2-bfc6-f545e7de58f7 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:25:57.020596 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d6f71f-4d3a-4951-ab57-7f4cea62f3e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:25:57.021244 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d6f71f-4d3a-4951-ab57-7f4cea62f3e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:25:57.021527 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d6f71f-4d3a-4951-ab57-7f4cea62f3e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:25:57.021806 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d6f71f-4d3a-4951-ab57-7f4cea62f3e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:25:57.022529 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d6f71f-4d3a-4951-ab57-7f4cea62f3e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:25:57.024217 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: WARNING py.warnings [None req-93d6f71f-4d3a-4951-ab57-7f4cea62f3e6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 17 19:25:57.024217 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: warnings.warn(msg) May 17 19:25:57.024217 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: May 17 19:25:57.025104 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d6f71f-4d3a-4951-ab57-7f4cea62f3e6 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:25:57.037498 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 750/1499] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:25:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:25:57.042265 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f931117a-6f5e-4171-b6b4-e83e0a2f1890 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:25:57.051839 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f931117a-6f5e-4171-b6b4-e83e0a2f1890 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:25:57.056365 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292b9e68-05a5-4bb7-8f36-6163858ff38b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:25:57.057002 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292b9e68-05a5-4bb7-8f36-6163858ff38b admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:25:57.057282 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292b9e68-05a5-4bb7-8f36-6163858ff38b admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:25:57.057555 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292b9e68-05a5-4bb7-8f36-6163858ff38b admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:25:57.058263 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292b9e68-05a5-4bb7-8f36-6163858ff38b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:25:57.059271 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: WARNING py.warnings [None req-292b9e68-05a5-4bb7-8f36-6163858ff38b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 17 19:25:57.059271 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: warnings.warn(msg) May 17 19:25:57.059271 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: May 17 19:25:57.060196 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292b9e68-05a5-4bb7-8f36-6163858ff38b admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:25:57.074155 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 750/1500] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:25:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:25:57.077930 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d6f71f-4d3a-4951-ab57-7f4cea62f3e6 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:25:57.088055 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d6f71f-4d3a-4951-ab57-7f4cea62f3e6 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:25:57.092466 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80523fa1-c98e-4157-8b1d-63cdd33efd88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:25:57.093329 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80523fa1-c98e-4157-8b1d-63cdd33efd88 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:25:57.093606 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80523fa1-c98e-4157-8b1d-63cdd33efd88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:25:57.093879 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80523fa1-c98e-4157-8b1d-63cdd33efd88 admin admin] PATH_INFO: `/v3/projects/a5adbd26418a4b6c8c54c0480a9b2c9b/users/0dc854c490dd483b9092c00a48d62278/roles/114e6a16e63447d1924707ace5aace25` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:25:57.108750 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80523fa1-c98e-4157-8b1d-63cdd33efd88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5adbd26418a4b6c8c54c0480a9b2c9b, user_id=0dc854c490dd483b9092c00a48d62278, role_id=114e6a16e63447d1924707ace5aace25)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:25:57.109773 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: WARNING py.warnings [None req-80523fa1-c98e-4157-8b1d-63cdd33efd88 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 17 19:25:57.109773 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: warnings.warn(msg) May 17 19:25:57.109773 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: May 17 19:25:57.111721 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80523fa1-c98e-4157-8b1d-63cdd33efd88 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:25:57.185181 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 751/1501] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:25:57 2022] PUT /identity/v3/projects/a5adbd26418a4b6c8c54c0480a9b2c9b/users/0dc854c490dd483b9092c00a48d62278/roles/114e6a16e63447d1924707ace5aace25 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:25:57.196312 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35959f20-3060-4694-af6d-b16db090f6c7 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:25:57.196604 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35959f20-3060-4694-af6d-b16db090f6c7 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:25:57.196874 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35959f20-3060-4694-af6d-b16db090f6c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:25:57.222295 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-35959f20-3060-4694-af6d-b16db090f6c7 None None] MFA Rules not processed for user `0dc854c490dd483b9092c00a48d62278`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:25:57.223759 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-35959f20-3060-4694-af6d-b16db090f6c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:25:57.285588 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 751/1502] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:25:57 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:25:57.299334 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25641f1-00af-4f9c-b475-3974706e7fc2 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:25:57.299663 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25641f1-00af-4f9c-b475-3974706e7fc2 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:25:57.300048 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25641f1-00af-4f9c-b475-3974706e7fc2 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:25:57.301450 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 752/1503] 10.222.0.58 () {54 vars in 963 bytes} [Tue May 17 19:25:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:25:57.306099 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbd3dc6-c7ce-4d87-9090-94ced28a36cc None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:25:57.306394 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbd3dc6-c7ce-4d87-9090-94ced28a36cc None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:25:57.306661 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbd3dc6-c7ce-4d87-9090-94ced28a36cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:25:57.324294 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-cbbd3dc6-c7ce-4d87-9090-94ced28a36cc None None] MFA Rules not processed for user `d05409b38ae346458abfe41ebda1c59e`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:25:57.329101 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-cbbd3dc6-c7ce-4d87-9090-94ced28a36cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:25:57.373341 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 752/1504] 10.222.0.58 () {58 vars in 1060 bytes} [Tue May 17 19:25:57 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:25:57.380593 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25641f1-00af-4f9c-b475-3974706e7fc2 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:25:57.391372 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25641f1-00af-4f9c-b475-3974706e7fc2 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:25:57.394920 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b19415-7f33-4329-a393-a75595d11efe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd05409b38ae346458abfe41ebda1c59e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:25:57.395564 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b19415-7f33-4329-a393-a75595d11efe service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:25:57.395845 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b19415-7f33-4329-a393-a75595d11efe service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:25:57.396114 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b19415-7f33-4329-a393-a75595d11efe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:25:57.404697 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b19415-7f33-4329-a393-a75595d11efe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:25:57.406109 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b19415-7f33-4329-a393-a75595d11efe service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:25:57.414516 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 753/1505] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:25:57 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 17 19:25:57.456204 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f308f0bb-07b0-4021-9064-5e8010d6de1f None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:25:57.456487 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f308f0bb-07b0-4021-9064-5e8010d6de1f None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:25:57.456757 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f308f0bb-07b0-4021-9064-5e8010d6de1f None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:25:57.458021 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 753/1506] 10.222.0.58 () {52 vars in 829 bytes} [Tue May 17 19:25:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:25:57.461624 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b19415-7f33-4329-a393-a75595d11efe service cinder] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:25:57.522773 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b19415-7f33-4329-a393-a75595d11efe service cinder] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:25:57.532664 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d49c9c-f488-48cd-8cb1-24e70925739a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:25:57.533289 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d49c9c-f488-48cd-8cb1-24e70925739a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:25:57.533561 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d49c9c-f488-48cd-8cb1-24e70925739a admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:25:57.533826 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d49c9c-f488-48cd-8cb1-24e70925739a admin admin] PATH_INFO: `/v3/users/0dc854c490dd483b9092c00a48d62278` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:25:57.534987 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d49c9c-f488-48cd-8cb1-24e70925739a admin admin] RBAC: Authorizing `identity:delete_user(user_id=0dc854c490dd483b9092c00a48d62278)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:25:57.536078 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: WARNING py.warnings [None req-b8d49c9c-f488-48cd-8cb1-24e70925739a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 17 19:25:57.536078 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: warnings.warn(msg) May 17 19:25:57.536078 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: May 17 19:25:57.537222 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d49c9c-f488-48cd-8cb1-24e70925739a admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:25:57.767764 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-b8d49c9c-f488-48cd-8cb1-24e70925739a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0dc854c490dd483b9092c00a48d62278'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:25:57.779933 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-b8d49c9c-f488-48cd-8cb1-24e70925739a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0dc854c490dd483b9092c00a48d62278'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:25:57.799190 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-b8d49c9c-f488-48cd-8cb1-24e70925739a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0dc854c490dd483b9092c00a48d62278'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:25:57.799936 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-b8d49c9c-f488-48cd-8cb1-24e70925739a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0dc854c490dd483b9092c00a48d62278'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:25:57.800567 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-b8d49c9c-f488-48cd-8cb1-24e70925739a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0dc854c490dd483b9092c00a48d62278'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:25:57.870729 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 754/1507] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 17 19:25:57 2022] DELETE /identity/v3/users/0dc854c490dd483b9092c00a48d62278 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:25:57.875199 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050331a5-dd16-4d66-9231-4e7ceeacbf3f None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:25:57.875705 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050331a5-dd16-4d66-9231-4e7ceeacbf3f None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:25:57.875813 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050331a5-dd16-4d66-9231-4e7ceeacbf3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:25:57.893256 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-050331a5-dd16-4d66-9231-4e7ceeacbf3f None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:25:57.894415 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-050331a5-dd16-4d66-9231-4e7ceeacbf3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:25:57.949044 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 754/1508] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:25:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:25:57.958923 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d49c9c-f488-48cd-8cb1-24e70925739a admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:25:57.960369 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-b8d49c9c-f488-48cd-8cb1-24e70925739a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:25:58.015829 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d49c9c-f488-48cd-8cb1-24e70925739a admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:25:58.019368 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6fd343-53cf-4eea-a337-bd42eb2c34bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:25:58.020028 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6fd343-53cf-4eea-a337-bd42eb2c34bc service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:25:58.020310 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6fd343-53cf-4eea-a337-bd42eb2c34bc service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:25:58.020585 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6fd343-53cf-4eea-a337-bd42eb2c34bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:25:58.029290 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6fd343-53cf-4eea-a337-bd42eb2c34bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:25:58.030705 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6fd343-53cf-4eea-a337-bd42eb2c34bc service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:25:58.039929 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 755/1509] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:25:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:25:58.529033 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050331a5-dd16-4d66-9231-4e7ceeacbf3f None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:25:58.530489 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-050331a5-dd16-4d66-9231-4e7ceeacbf3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:25:58.542679 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050331a5-dd16-4d66-9231-4e7ceeacbf3f None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:25:58.547207 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e44016-e927-400b-9d2d-41c7ee422730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:25:58.548088 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e44016-e927-400b-9d2d-41c7ee422730 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:25:58.548359 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e44016-e927-400b-9d2d-41c7ee422730 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:25:58.548635 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e44016-e927-400b-9d2d-41c7ee422730 admin admin] PATH_INFO: `/v3/projects/a5adbd26418a4b6c8c54c0480a9b2c9b` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:25:58.549894 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e44016-e927-400b-9d2d-41c7ee422730 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5adbd26418a4b6c8c54c0480a9b2c9b)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:25:58.551283 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: WARNING py.warnings [None req-49e44016-e927-400b-9d2d-41c7ee422730 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 17 19:25:58.551283 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: warnings.warn(msg) May 17 19:25:58.551283 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: May 17 19:25:58.552130 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e44016-e927-400b-9d2d-41c7ee422730 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:25:58.663432 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-49e44016-e927-400b-9d2d-41c7ee422730 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5adbd26418a4b6c8c54c0480a9b2c9b', 'request_id': 'req-49e44016-e927-400b-9d2d-41c7ee422730'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:25:58.669966 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-49e44016-e927-400b-9d2d-41c7ee422730 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5adbd26418a4b6c8c54c0480a9b2c9b', 'request_id': 'req-49e44016-e927-400b-9d2d-41c7ee422730'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:25:58.778999 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-49e44016-e927-400b-9d2d-41c7ee422730 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5adbd26418a4b6c8c54c0480a9b2c9b', 'request_id': 'req-49e44016-e927-400b-9d2d-41c7ee422730'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:25:58.797414 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-49e44016-e927-400b-9d2d-41c7ee422730 admin admin] The token cache is being invalidate because project a5adbd26418a4b6c8c54c0480a9b2c9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 17 19:25:58.797698 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-49e44016-e927-400b-9d2d-41c7ee422730 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:25:58.798406 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-49e44016-e927-400b-9d2d-41c7ee422730 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:25:58.799968 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 755/1510] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 17 19:25:58 2022] DELETE /identity/v3/projects/a5adbd26418a4b6c8c54c0480a9b2c9b => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:27:37.808697 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397ff918-0b52-41c6-ade8-d85fbca78fed None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:27:37.809303 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397ff918-0b52-41c6-ade8-d85fbca78fed None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:27:37.809303 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397ff918-0b52-41c6-ade8-d85fbca78fed None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:27:37.810677 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 756/1511] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 17 19:27:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:27:37.816019 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee26d8c-70b3-4ca3-bcac-5a7420e8023e None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:27:37.816452 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee26d8c-70b3-4ca3-bcac-5a7420e8023e None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:27:37.816557 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee26d8c-70b3-4ca3-bcac-5a7420e8023e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:27:37.834534 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-dee26d8c-70b3-4ca3-bcac-5a7420e8023e None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:27:37.835766 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-dee26d8c-70b3-4ca3-bcac-5a7420e8023e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:27:37.890825 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 756/1512] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 17 19:27:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:27:38.055163 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397ff918-0b52-41c6-ade8-d85fbca78fed None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:27:38.056898 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-397ff918-0b52-41c6-ade8-d85fbca78fed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:27:38.130070 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397ff918-0b52-41c6-ade8-d85fbca78fed None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:27:38.135883 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d174c6-13aa-455a-8df4-937648d1044f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c1f878b0dc4d63a4f5dcb12fac1dc1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:27:38.136556 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d174c6-13aa-455a-8df4-937648d1044f service nova] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:27:38.136806 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d174c6-13aa-455a-8df4-937648d1044f service nova] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:27:38.137077 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d174c6-13aa-455a-8df4-937648d1044f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:27:38.145093 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d174c6-13aa-455a-8df4-937648d1044f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:27:38.146671 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d174c6-13aa-455a-8df4-937648d1044f service nova] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:27:38.155853 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 757/1513] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:27:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:27:39.066883 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93077760-ebfe-4945-b672-7f206079302d None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:27:39.067399 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93077760-ebfe-4945-b672-7f206079302d None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:27:39.067509 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93077760-ebfe-4945-b672-7f206079302d None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:27:39.068892 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 757/1514] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 17 19:27:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:27:39.073710 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2564d7-9ead-4d4f-a325-8299b5a66cde None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:27:39.074146 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2564d7-9ead-4d4f-a325-8299b5a66cde None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:27:39.074333 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2564d7-9ead-4d4f-a325-8299b5a66cde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:27:39.092165 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-bf2564d7-9ead-4d4f-a325-8299b5a66cde None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:27:39.093465 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-bf2564d7-9ead-4d4f-a325-8299b5a66cde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:27:39.102799 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 758/1515] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 17 19:27:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:27:39.139067 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283995ac-0f8e-4a8f-9f32-32ead3459a3e None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:27:39.139351 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283995ac-0f8e-4a8f-9f32-32ead3459a3e None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:27:39.139657 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283995ac-0f8e-4a8f-9f32-32ead3459a3e None None] PATH_INFO: `/v3` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:27:39.141077 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 758/1516] 10.222.0.69 () {54 vars in 925 bytes} [Tue May 17 19:27:39 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:27:39.147073 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ff9c73-ce9a-422f-a3c7-7efa65e2250f None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:27:39.147434 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ff9c73-ce9a-422f-a3c7-7efa65e2250f None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:27:39.147818 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ff9c73-ce9a-422f-a3c7-7efa65e2250f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:27:39.166340 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-40ff9c73-ce9a-422f-a3c7-7efa65e2250f None None] MFA Rules not processed for user `06538aab5f194af7b7c677d6b94aef2a`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:27:39.167556 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-40ff9c73-ce9a-422f-a3c7-7efa65e2250f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:27:39.230670 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 759/1517] 10.222.0.69 () {58 vars in 1015 bytes} [Tue May 17 19:27:39 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:27:39.255804 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283995ac-0f8e-4a8f-9f32-32ead3459a3e None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:27:39.269849 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283995ac-0f8e-4a8f-9f32-32ead3459a3e None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:27:39.275193 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3825e7-e857-4750-bc3d-2e6dcd4894fc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c1f878b0dc4d63a4f5dcb12fac1dc1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:27:39.275855 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3825e7-e857-4750-bc3d-2e6dcd4894fc service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:27:39.276141 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3825e7-e857-4750-bc3d-2e6dcd4894fc service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:27:39.276414 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3825e7-e857-4750-bc3d-2e6dcd4894fc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:27:39.284966 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3825e7-e857-4750-bc3d-2e6dcd4894fc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:27:39.286590 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3825e7-e857-4750-bc3d-2e6dcd4894fc service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:27:39.295747 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 759/1518] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:27:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:27:43.386854 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2d2859-e1d3-4980-85f4-c518f1c0b602 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:27:43.387318 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2d2859-e1d3-4980-85f4-c518f1c0b602 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:27:43.387428 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2d2859-e1d3-4980-85f4-c518f1c0b602 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:27:43.388798 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 760/1519] 10.222.0.58 () {54 vars in 960 bytes} [Tue May 17 19:27:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:27:43.393831 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54edc171-c4f5-422d-bbf4-881c4aabea05 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:27:43.394251 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54edc171-c4f5-422d-bbf4-881c4aabea05 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:27:43.394382 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54edc171-c4f5-422d-bbf4-881c4aabea05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:27:43.413626 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-54edc171-c4f5-422d-bbf4-881c4aabea05 None None] MFA Rules not processed for user `06538aab5f194af7b7c677d6b94aef2a`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:27:43.414815 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-54edc171-c4f5-422d-bbf4-881c4aabea05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:27:43.423141 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 760/1520] 10.222.0.58 () {58 vars in 1057 bytes} [Tue May 17 19:27:43 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:27:43.432459 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2d2859-e1d3-4980-85f4-c518f1c0b602 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:27:43.446131 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2d2859-e1d3-4980-85f4-c518f1c0b602 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:27:43.451037 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c794c458-04b5-4d86-9c1c-c0c45529fca8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06538aab5f194af7b7c677d6b94aef2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:27:43.451689 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c794c458-04b5-4d86-9c1c-c0c45529fca8 service placement] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:27:43.451974 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c794c458-04b5-4d86-9c1c-c0c45529fca8 service placement] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:27:43.452246 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c794c458-04b5-4d86-9c1c-c0c45529fca8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:27:43.460244 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c794c458-04b5-4d86-9c1c-c0c45529fca8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:27:43.461818 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c794c458-04b5-4d86-9c1c-c0c45529fca8 service placement] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:27:43.471007 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 761/1521] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:27:43 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:27:50.591714 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d360a8f7-113d-42bb-a5f7-bf5de0e30926 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:27:50.592283 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d360a8f7-113d-42bb-a5f7-bf5de0e30926 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:27:50.592283 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d360a8f7-113d-42bb-a5f7-bf5de0e30926 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:27:50.593683 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 761/1522] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 17 19:27:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:27:50.599296 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75be649-d3aa-4c42-8e12-32d3ea6474bd None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:27:50.599789 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75be649-d3aa-4c42-8e12-32d3ea6474bd None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:27:50.599889 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75be649-d3aa-4c42-8e12-32d3ea6474bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:27:50.617261 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-b75be649-d3aa-4c42-8e12-32d3ea6474bd None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:27:50.618453 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-b75be649-d3aa-4c42-8e12-32d3ea6474bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:27:50.626147 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 762/1523] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 17 19:27:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:27:50.775616 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d360a8f7-113d-42bb-a5f7-bf5de0e30926 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:27:50.789944 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d360a8f7-113d-42bb-a5f7-bf5de0e30926 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:27:50.795442 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7f6414-4c4e-4306-b270-f53da1641f88 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c1f878b0dc4d63a4f5dcb12fac1dc1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:27:50.796121 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7f6414-4c4e-4306-b270-f53da1641f88 service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:27:50.796398 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7f6414-4c4e-4306-b270-f53da1641f88 service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:27:50.796665 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7f6414-4c4e-4306-b270-f53da1641f88 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:27:50.804782 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7f6414-4c4e-4306-b270-f53da1641f88 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:27:50.806358 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7f6414-4c4e-4306-b270-f53da1641f88 service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:27:50.815470 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 762/1524] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:27:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:27:51.718251 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863f9a58-fc61-4f08-9311-f768063bb337 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:27:51.718845 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863f9a58-fc61-4f08-9311-f768063bb337 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:27:51.718845 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863f9a58-fc61-4f08-9311-f768063bb337 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:27:51.720307 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 763/1525] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 17 19:27:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:27:51.725689 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e942cab-5341-4588-846a-ec6bc8a35c48 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:27:51.726089 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e942cab-5341-4588-846a-ec6bc8a35c48 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:27:51.726241 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e942cab-5341-4588-846a-ec6bc8a35c48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:27:51.744422 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-8e942cab-5341-4588-846a-ec6bc8a35c48 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:27:51.745609 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-8e942cab-5341-4588-846a-ec6bc8a35c48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:27:51.753344 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 763/1526] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 17 19:27:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:27:51.904945 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863f9a58-fc61-4f08-9311-f768063bb337 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:27:51.919281 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863f9a58-fc61-4f08-9311-f768063bb337 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:27:51.924538 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dd334b-1ea5-4a33-85e2-776d30d47869 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c1f878b0dc4d63a4f5dcb12fac1dc1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:27:51.925153 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dd334b-1ea5-4a33-85e2-776d30d47869 service nova] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:27:51.925427 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dd334b-1ea5-4a33-85e2-776d30d47869 service nova] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:27:51.926377 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dd334b-1ea5-4a33-85e2-776d30d47869 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:27:51.934162 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dd334b-1ea5-4a33-85e2-776d30d47869 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:27:51.935768 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dd334b-1ea5-4a33-85e2-776d30d47869 service nova] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:27:51.944828 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 764/1527] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:27:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:27:55.941304 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f779424-afe2-4963-b858-75bd5e696e05 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:27:55.941879 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f779424-afe2-4963-b858-75bd5e696e05 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:27:55.941879 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f779424-afe2-4963-b858-75bd5e696e05 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:27:55.943213 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 764/1528] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 17 19:27:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:27:55.948290 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aca9bc9-0cbe-4f9a-aed8-e91a5d79cd0d None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:27:55.948674 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aca9bc9-0cbe-4f9a-aed8-e91a5d79cd0d None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:27:55.948841 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aca9bc9-0cbe-4f9a-aed8-e91a5d79cd0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:27:55.966592 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-5aca9bc9-0cbe-4f9a-aed8-e91a5d79cd0d None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:27:55.967802 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-5aca9bc9-0cbe-4f9a-aed8-e91a5d79cd0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:27:55.976140 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 765/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 17 19:27:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:27:56.011126 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f779424-afe2-4963-b858-75bd5e696e05 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:27:56.012815 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-0f779424-afe2-4963-b858-75bd5e696e05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:27:56.068799 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f779424-afe2-4963-b858-75bd5e696e05 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:27:56.072278 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e39b97-e34a-4461-aebe-517f2220d3d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:27:56.072908 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e39b97-e34a-4461-aebe-517f2220d3d6 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:27:56.073183 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e39b97-e34a-4461-aebe-517f2220d3d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:27:56.073452 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e39b97-e34a-4461-aebe-517f2220d3d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:27:56.082465 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e39b97-e34a-4461-aebe-517f2220d3d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:27:56.083907 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e39b97-e34a-4461-aebe-517f2220d3d6 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:27:56.092797 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 765/1530] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:27:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:27:58.275013 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5d61c5-5591-4249-8185-94ba182a0275 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:27:58.275592 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5d61c5-5591-4249-8185-94ba182a0275 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:27:58.275592 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5d61c5-5591-4249-8185-94ba182a0275 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:27:58.276948 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 766/1531] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 17 19:27:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:27:58.282132 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bb9eb5-0d36-41bd-8201-696a2e86a669 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:27:58.282562 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bb9eb5-0d36-41bd-8201-696a2e86a669 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:27:58.282701 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bb9eb5-0d36-41bd-8201-696a2e86a669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:27:58.300637 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-21bb9eb5-0d36-41bd-8201-696a2e86a669 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:27:58.301807 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-21bb9eb5-0d36-41bd-8201-696a2e86a669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:27:58.309298 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 766/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 17 19:27:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:27:58.343733 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5d61c5-5591-4249-8185-94ba182a0275 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:27:58.345405 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-7f5d61c5-5591-4249-8185-94ba182a0275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:27:58.408845 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5d61c5-5591-4249-8185-94ba182a0275 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:27:58.412418 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d46bc45-dfbb-4569-bb82-ca968aec7978 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:27:58.413050 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d46bc45-dfbb-4569-bb82-ca968aec7978 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:27:58.413329 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d46bc45-dfbb-4569-bb82-ca968aec7978 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:27:58.413612 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d46bc45-dfbb-4569-bb82-ca968aec7978 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:27:58.422388 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d46bc45-dfbb-4569-bb82-ca968aec7978 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:27:58.423848 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d46bc45-dfbb-4569-bb82-ca968aec7978 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:27:58.433477 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 767/1533] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:27:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 17 19:28:02.178463 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c0e166-eebe-4e25-9631-9fcf3fd4d464 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:28:02.178992 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c0e166-eebe-4e25-9631-9fcf3fd4d464 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:28:02.179113 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c0e166-eebe-4e25-9631-9fcf3fd4d464 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:28:02.180492 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 767/1534] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 17 19:28:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:28:02.185677 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f49f800-f11d-4dc9-889c-c0499ee218b6 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:28:02.191611 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f49f800-f11d-4dc9-889c-c0499ee218b6 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:28:02.191611 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f49f800-f11d-4dc9-889c-c0499ee218b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:28:02.206100 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-9f49f800-f11d-4dc9-889c-c0499ee218b6 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:28:02.208217 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-9f49f800-f11d-4dc9-889c-c0499ee218b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:28:02.217383 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 768/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 17 19:28:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:28:02.489544 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c0e166-eebe-4e25-9631-9fcf3fd4d464 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:28:02.491348 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-e7c0e166-eebe-4e25-9631-9fcf3fd4d464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:28:02.551192 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c0e166-eebe-4e25-9631-9fcf3fd4d464 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:28:02.554732 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308dafbe-1caf-470e-aeb6-9ed27fd56b30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd05409b38ae346458abfe41ebda1c59e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:28:02.555367 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308dafbe-1caf-470e-aeb6-9ed27fd56b30 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:28:02.555684 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308dafbe-1caf-470e-aeb6-9ed27fd56b30 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:28:02.555959 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308dafbe-1caf-470e-aeb6-9ed27fd56b30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:28:02.564644 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308dafbe-1caf-470e-aeb6-9ed27fd56b30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:28:02.566597 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308dafbe-1caf-470e-aeb6-9ed27fd56b30 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:28:02.575821 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 768/1536] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:28:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:28:04.813162 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295c72af-ef3a-4c9b-8a21-b81c477bda5c None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:28:04.813764 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295c72af-ef3a-4c9b-8a21-b81c477bda5c None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:28:04.813764 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295c72af-ef3a-4c9b-8a21-b81c477bda5c None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:28:04.815107 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 769/1537] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 17 19:28:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:28:04.820823 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fecd65-10f1-4363-99b6-801f1e26d30c None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:28:04.821277 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fecd65-10f1-4363-99b6-801f1e26d30c None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:28:04.821396 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fecd65-10f1-4363-99b6-801f1e26d30c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:28:04.839559 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-48fecd65-10f1-4363-99b6-801f1e26d30c None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:28:04.840745 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-48fecd65-10f1-4363-99b6-801f1e26d30c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:28:04.848761 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 769/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 17 19:28:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:28:05.118086 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295c72af-ef3a-4c9b-8a21-b81c477bda5c None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:28:05.119770 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-295c72af-ef3a-4c9b-8a21-b81c477bda5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:28:05.132402 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295c72af-ef3a-4c9b-8a21-b81c477bda5c None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:28:05.135844 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e71415-0300-44af-bfbc-4c4626b949b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd05409b38ae346458abfe41ebda1c59e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:28:05.136466 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e71415-0300-44af-bfbc-4c4626b949b9 service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:28:05.136746 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e71415-0300-44af-bfbc-4c4626b949b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:28:05.137019 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e71415-0300-44af-bfbc-4c4626b949b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:28:05.146052 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e71415-0300-44af-bfbc-4c4626b949b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:28:05.147456 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e71415-0300-44af-bfbc-4c4626b949b9 service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:28:05.156529 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 770/1539] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:28:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:29:35.509079 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9e99ca-004d-449a-aab3-4c9851d4a94f None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:35.509709 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9e99ca-004d-449a-aab3-4c9851d4a94f None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:35.509709 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9e99ca-004d-449a-aab3-4c9851d4a94f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:35.535382 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-1a9e99ca-004d-449a-aab3-4c9851d4a94f None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:29:35.536607 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-1a9e99ca-004d-449a-aab3-4c9851d4a94f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:29:35.547996 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 770/1540] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:29:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:29:35.558877 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991bcaa9-84f1-4129-ab34-8cdae80041ae None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:35.559917 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991bcaa9-84f1-4129-ab34-8cdae80041ae None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:35.560410 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991bcaa9-84f1-4129-ab34-8cdae80041ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:35.587231 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-991bcaa9-84f1-4129-ab34-8cdae80041ae None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:29:35.589868 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-991bcaa9-84f1-4129-ab34-8cdae80041ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:29:35.606222 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 771/1541] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:29:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:29:35.610996 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9e99ca-004d-449a-aab3-4c9851d4a94f None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:35.621673 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9e99ca-004d-449a-aab3-4c9851d4a94f None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:35.632819 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6028634b-6189-498c-9dfa-4bd6362a6b16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:35.633506 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6028634b-6189-498c-9dfa-4bd6362a6b16 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:35.633789 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6028634b-6189-498c-9dfa-4bd6362a6b16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:35.634064 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6028634b-6189-498c-9dfa-4bd6362a6b16 admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:35.634795 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6028634b-6189-498c-9dfa-4bd6362a6b16 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:29:35.635100 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6028634b-6189-498c-9dfa-4bd6362a6b16 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:35.636780 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6028634b-6189-498c-9dfa-4bd6362a6b16 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:35.656290 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 771/1542] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 17 19:29:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:29:35.665629 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991bcaa9-84f1-4129-ab34-8cdae80041ae None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:35.678779 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991bcaa9-84f1-4129-ab34-8cdae80041ae None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:35.688021 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f41941-5ae8-4992-9bcf-8f84ba2b6bdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:35.689100 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f41941-5ae8-4992-9bcf-8f84ba2b6bdc admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:35.689591 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f41941-5ae8-4992-9bcf-8f84ba2b6bdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:35.690069 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f41941-5ae8-4992-9bcf-8f84ba2b6bdc admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:35.690513 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379ee39e-1c22-4122-84c7-02ec779ba8d4 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:35.690983 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f41941-5ae8-4992-9bcf-8f84ba2b6bdc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:35.691310 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379ee39e-1c22-4122-84c7-02ec779ba8d4 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:35.691837 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379ee39e-1c22-4122-84c7-02ec779ba8d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:35.692939 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f41941-5ae8-4992-9bcf-8f84ba2b6bdc admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:35.711794 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-379ee39e-1c22-4122-84c7-02ec779ba8d4 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:29:35.713977 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-379ee39e-1c22-4122-84c7-02ec779ba8d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:29:35.722975 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 772/1543] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:29:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:29:35.762106 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160f5b2c-a57f-4fd6-b4ed-dbf6e1091587 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:35.762732 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160f5b2c-a57f-4fd6-b4ed-dbf6e1091587 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:35.763261 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160f5b2c-a57f-4fd6-b4ed-dbf6e1091587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:35.797029 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-160f5b2c-a57f-4fd6-b4ed-dbf6e1091587 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:29:35.798618 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-160f5b2c-a57f-4fd6-b4ed-dbf6e1091587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:29:35.818407 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 772/1544] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 17 19:29:35 2022] POST /identity/v3/projects => generated 373 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:29:35.824672 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f41941-5ae8-4992-9bcf-8f84ba2b6bdc admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:35.827132 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 773/1545] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:29:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 17 19:29:35.832718 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160f5b2c-a57f-4fd6-b4ed-dbf6e1091587 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:35.847528 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160f5b2c-a57f-4fd6-b4ed-dbf6e1091587 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:35.968224 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f41941-5ae8-4992-9bcf-8f84ba2b6bdc admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:35.975450 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f1e98a-1da5-4854-8009-099740940b13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:35.976827 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f1e98a-1da5-4854-8009-099740940b13 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:35.976827 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f1e98a-1da5-4854-8009-099740940b13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:35.977047 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f1e98a-1da5-4854-8009-099740940b13 admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:35.978517 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f1e98a-1da5-4854-8009-099740940b13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:35.979069 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bba888c-f4d0-4ad2-befa-8b5018443534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:35.979775 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: WARNING py.warnings [None req-c8f1e98a-1da5-4854-8009-099740940b13 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 17 19:29:35.979775 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: warnings.warn(msg) May 17 19:29:35.979775 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: May 17 19:29:35.980429 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bba888c-f4d0-4ad2-befa-8b5018443534 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:35.980858 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bba888c-f4d0-4ad2-befa-8b5018443534 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:35.981043 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f1e98a-1da5-4854-8009-099740940b13 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:35.981442 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bba888c-f4d0-4ad2-befa-8b5018443534 admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:35.982345 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bba888c-f4d0-4ad2-befa-8b5018443534 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:29:35.982875 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bba888c-f4d0-4ad2-befa-8b5018443534 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:35.985059 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bba888c-f4d0-4ad2-befa-8b5018443534 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:35.998872 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 774/1546] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 17 19:29:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:29:36.001152 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4ec68b-9a99-4094-aab5-afa83c9f2394 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:36.001633 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4ec68b-9a99-4094-aab5-afa83c9f2394 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:36.002058 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4ec68b-9a99-4094-aab5-afa83c9f2394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:36.024506 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-db4ec68b-9a99-4094-aab5-afa83c9f2394 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:29:36.025675 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-db4ec68b-9a99-4094-aab5-afa83c9f2394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:29:36.034088 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 775/1547] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:29:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:29:36.035278 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dfc407-c489-4df6-a4a3-79dd131078cc None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:36.035592 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dfc407-c489-4df6-a4a3-79dd131078cc None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:36.035863 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dfc407-c489-4df6-a4a3-79dd131078cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:36.054128 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-b9dfc407-c489-4df6-a4a3-79dd131078cc None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:29:36.055297 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b9dfc407-c489-4df6-a4a3-79dd131078cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:29:36.059469 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 773/1548] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 17 19:29:35 2022] POST /identity/v3/users => generated 287 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:29:36.061058 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599341aa-9673-4377-9b5e-6691407b2694 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:36.062295 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599341aa-9673-4377-9b5e-6691407b2694 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:36.062295 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599341aa-9673-4377-9b5e-6691407b2694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:36.064595 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 776/1549] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:29:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:29:36.070715 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbdd1e3-e3f5-48d2-b1b2-7067e19d13ba None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:36.071230 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbdd1e3-e3f5-48d2-b1b2-7067e19d13ba None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:36.071693 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbdd1e3-e3f5-48d2-b1b2-7067e19d13ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:36.082792 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-599341aa-9673-4377-9b5e-6691407b2694 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:29:36.084038 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-599341aa-9673-4377-9b5e-6691407b2694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:29:36.092539 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 774/1550] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:29:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:29:36.094845 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599341aa-9673-4377-9b5e-6691407b2694 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:36.097182 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-1fbdd1e3-e3f5-48d2-b1b2-7067e19d13ba None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:29:36.098700 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-1fbdd1e3-e3f5-48d2-b1b2-7067e19d13ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:29:36.109833 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599341aa-9673-4377-9b5e-6691407b2694 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:36.110258 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 777/1551] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:29:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:29:36.111898 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebdef94-0a35-40ef-a9ca-4c74a3d05034 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:36.112179 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebdef94-0a35-40ef-a9ca-4c74a3d05034 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:36.112453 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebdef94-0a35-40ef-a9ca-4c74a3d05034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:36.117245 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19902aba-2ec5-4f3c-80fe-4fa5e4fcf5c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:36.118203 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19902aba-2ec5-4f3c-80fe-4fa5e4fcf5c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:36.118720 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19902aba-2ec5-4f3c-80fe-4fa5e4fcf5c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:36.119216 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19902aba-2ec5-4f3c-80fe-4fa5e4fcf5c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:36.120270 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19902aba-2ec5-4f3c-80fe-4fa5e4fcf5c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:36.122251 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19902aba-2ec5-4f3c-80fe-4fa5e4fcf5c7 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:36.130084 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-6ebdef94-0a35-40ef-a9ca-4c74a3d05034 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:29:36.131256 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-6ebdef94-0a35-40ef-a9ca-4c74a3d05034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:29:36.139099 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 778/1552] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:29:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:29:36.140479 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1be13ad-3534-4fe5-a4cf-f22f174f8823 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:36.140754 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1be13ad-3534-4fe5-a4cf-f22f174f8823 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:36.141018 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1be13ad-3534-4fe5-a4cf-f22f174f8823 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:36.158816 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-f1be13ad-3534-4fe5-a4cf-f22f174f8823 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:29:36.160014 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-f1be13ad-3534-4fe5-a4cf-f22f174f8823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:29:36.167623 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 779/1553] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:29:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:29:36.168379 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1be13ad-3534-4fe5-a4cf-f22f174f8823 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:36.177571 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1be13ad-3534-4fe5-a4cf-f22f174f8823 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:36.181990 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f80395e-8531-4c83-af6a-80ca3dc6f865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:36.182602 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f80395e-8531-4c83-af6a-80ca3dc6f865 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:36.182876 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f80395e-8531-4c83-af6a-80ca3dc6f865 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:36.183144 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f80395e-8531-4c83-af6a-80ca3dc6f865 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:36.183869 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f80395e-8531-4c83-af6a-80ca3dc6f865 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:36.185480 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f80395e-8531-4c83-af6a-80ca3dc6f865 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:36.194973 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 780/1554] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:29:36 2022] GET /identity/v3/roles => generated 1420 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:29:36.200361 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3282bf-0213-480e-a08c-7a7540190bbf None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:36.200361 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3282bf-0213-480e-a08c-7a7540190bbf None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:36.200361 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3282bf-0213-480e-a08c-7a7540190bbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:36.213156 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-9a3282bf-0213-480e-a08c-7a7540190bbf None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:29:36.214304 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-9a3282bf-0213-480e-a08c-7a7540190bbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:29:36.222692 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 781/1555] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:29:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:29:36.223950 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b28e16-5b65-4f78-a648-ffd6105ffa79 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:36.224233 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b28e16-5b65-4f78-a648-ffd6105ffa79 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:36.224499 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b28e16-5b65-4f78-a648-ffd6105ffa79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:36.229485 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 775/1556] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 17 19:29:36 2022] POST /identity/v3/projects => generated 377 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:29:36.231020 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf8db20-6934-454c-bc92-996467fea3ca None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:36.231335 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf8db20-6934-454c-bc92-996467fea3ca None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:36.231642 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf8db20-6934-454c-bc92-996467fea3ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:36.242530 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-01b28e16-5b65-4f78-a648-ffd6105ffa79 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:29:36.243728 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-01b28e16-5b65-4f78-a648-ffd6105ffa79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:29:36.249969 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-dcf8db20-6934-454c-bc92-996467fea3ca None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:29:36.251223 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-dcf8db20-6934-454c-bc92-996467fea3ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:29:36.303938 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 782/1557] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:29:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:29:36.305307 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c8c682-b2d4-4f55-8b52-2a5d585f3399 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:36.305584 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c8c682-b2d4-4f55-8b52-2a5d585f3399 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:36.305886 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c8c682-b2d4-4f55-8b52-2a5d585f3399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:36.311369 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 776/1558] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:29:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:29:36.312802 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b3cc48-82f4-45ae-86c3-c1114d17281a None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:36.313101 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b3cc48-82f4-45ae-86c3-c1114d17281a None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:36.313373 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b3cc48-82f4-45ae-86c3-c1114d17281a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:36.324150 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-b7c8c682-b2d4-4f55-8b52-2a5d585f3399 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:29:36.325386 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b7c8c682-b2d4-4f55-8b52-2a5d585f3399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:29:36.331333 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-87b3cc48-82f4-45ae-86c3-c1114d17281a None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:29:36.332521 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-87b3cc48-82f4-45ae-86c3-c1114d17281a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:29:36.333911 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 783/1559] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:29:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:29:36.334687 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c8c682-b2d4-4f55-8b52-2a5d585f3399 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:36.340655 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 777/1560] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:29:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:29:36.341603 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b3cc48-82f4-45ae-86c3-c1114d17281a None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:36.344971 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c8c682-b2d4-4f55-8b52-2a5d585f3399 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:36.349808 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d37ca0-6cc2-4985-a314-10937696e45b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:36.350447 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d37ca0-6cc2-4985-a314-10937696e45b admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:36.350726 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d37ca0-6cc2-4985-a314-10937696e45b admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:36.350998 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d37ca0-6cc2-4985-a314-10937696e45b admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:36.351774 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d37ca0-6cc2-4985-a314-10937696e45b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:29:36.352077 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d37ca0-6cc2-4985-a314-10937696e45b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:36.353038 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b3cc48-82f4-45ae-86c3-c1114d17281a None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:36.353684 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d37ca0-6cc2-4985-a314-10937696e45b admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:36.358338 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf6466f-7c8d-4d3f-9717-0526ddfd8246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:36.359064 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf6466f-7c8d-4d3f-9717-0526ddfd8246 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:36.359345 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf6466f-7c8d-4d3f-9717-0526ddfd8246 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:36.359700 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf6466f-7c8d-4d3f-9717-0526ddfd8246 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:36.360428 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf6466f-7c8d-4d3f-9717-0526ddfd8246 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:36.362003 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf6466f-7c8d-4d3f-9717-0526ddfd8246 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:36.365272 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 784/1561] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 17 19:29:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:29:36.366143 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d37ca0-6cc2-4985-a314-10937696e45b admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:36.375480 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d37ca0-6cc2-4985-a314-10937696e45b admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:36.380385 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 778/1562] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:29:36 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:29:36.380385 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf6466f-7c8d-4d3f-9717-0526ddfd8246 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:36.382163 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd70c1d-4a74-401b-a482-c27a03187796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:36.382797 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd70c1d-4a74-401b-a482-c27a03187796 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:36.383077 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd70c1d-4a74-401b-a482-c27a03187796 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:36.383354 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd70c1d-4a74-401b-a482-c27a03187796 admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:36.384090 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd70c1d-4a74-401b-a482-c27a03187796 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:29:36.384395 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd70c1d-4a74-401b-a482-c27a03187796 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:36.385983 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd70c1d-4a74-401b-a482-c27a03187796 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:36.387598 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf6466f-7c8d-4d3f-9717-0526ddfd8246 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:36.392670 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135573c5-1571-464a-9119-0e40a82fd8ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:36.393298 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135573c5-1571-464a-9119-0e40a82fd8ca admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:36.393573 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135573c5-1571-464a-9119-0e40a82fd8ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:36.393843 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135573c5-1571-464a-9119-0e40a82fd8ca admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:36.394508 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135573c5-1571-464a-9119-0e40a82fd8ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:36.396408 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135573c5-1571-464a-9119-0e40a82fd8ca admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:36.398557 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 785/1563] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 17 19:29:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:29:36.399225 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd70c1d-4a74-401b-a482-c27a03187796 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:36.410533 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd70c1d-4a74-401b-a482-c27a03187796 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:36.415187 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f28dde-8899-4aa0-b2d6-e60d7e9763c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:36.415848 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f28dde-8899-4aa0-b2d6-e60d7e9763c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:36.418174 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f28dde-8899-4aa0-b2d6-e60d7e9763c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:36.418174 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f28dde-8899-4aa0-b2d6-e60d7e9763c5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:36.418174 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f28dde-8899-4aa0-b2d6-e60d7e9763c5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:29:36.418174 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f28dde-8899-4aa0-b2d6-e60d7e9763c5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:36.419142 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f28dde-8899-4aa0-b2d6-e60d7e9763c5 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:36.430393 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 786/1564] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 17 19:29:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:29:36.431781 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4429ed-aaf3-4431-9cb1-ce5f77d1adee None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:36.432075 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4429ed-aaf3-4431-9cb1-ce5f77d1adee None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:36.432335 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4429ed-aaf3-4431-9cb1-ce5f77d1adee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:36.449167 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-cc4429ed-aaf3-4431-9cb1-ce5f77d1adee None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:29:36.450334 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-cc4429ed-aaf3-4431-9cb1-ce5f77d1adee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:29:36.458723 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 787/1565] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:29:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:29:36.464064 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4429ed-aaf3-4431-9cb1-ce5f77d1adee None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:36.469144 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4429ed-aaf3-4431-9cb1-ce5f77d1adee None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:36.473861 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7733be-dda4-4d39-9383-27ba80bd1d11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:36.474468 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7733be-dda4-4d39-9383-27ba80bd1d11 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:36.474744 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7733be-dda4-4d39-9383-27ba80bd1d11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:36.475014 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7733be-dda4-4d39-9383-27ba80bd1d11 admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:36.475750 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7733be-dda4-4d39-9383-27ba80bd1d11 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:29:36.476049 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7733be-dda4-4d39-9383-27ba80bd1d11 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:36.477614 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7733be-dda4-4d39-9383-27ba80bd1d11 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:36.486687 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 779/1566] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 17 19:29:36 2022] POST /identity/v3/users => generated 289 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:29:36.499917 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135573c5-1571-464a-9119-0e40a82fd8ca admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:36.500065 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 788/1567] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 17 19:29:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:29:36.500065 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7733be-dda4-4d39-9383-27ba80bd1d11 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:36.500252 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135573c5-1571-464a-9119-0e40a82fd8ca admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:36.500345 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7733be-dda4-4d39-9383-27ba80bd1d11 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:36.504517 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdf72bf-06cc-4e11-89b7-a7449c448261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:36.504766 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17b12c6-f21f-4721-bec6-cf2fa8d0cdb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:36.505168 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdf72bf-06cc-4e11-89b7-a7449c448261 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:36.505396 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17b12c6-f21f-4721-bec6-cf2fa8d0cdb6 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:36.505520 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdf72bf-06cc-4e11-89b7-a7449c448261 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:36.505674 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17b12c6-f21f-4721-bec6-cf2fa8d0cdb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:36.505806 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdf72bf-06cc-4e11-89b7-a7449c448261 admin admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:36.505946 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17b12c6-f21f-4721-bec6-cf2fa8d0cdb6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:36.506413 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdf72bf-06cc-4e11-89b7-a7449c448261 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:29:36.506612 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17b12c6-f21f-4721-bec6-cf2fa8d0cdb6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:36.506752 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdf72bf-06cc-4e11-89b7-a7449c448261 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:36.507650 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: WARNING py.warnings [None req-f17b12c6-f21f-4721-bec6-cf2fa8d0cdb6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 17 19:29:36.507650 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: warnings.warn(msg) May 17 19:29:36.507650 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: May 17 19:29:36.507907 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: WARNING py.warnings [None req-fcdf72bf-06cc-4e11-89b7-a7449c448261 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 17 19:29:36.507907 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: warnings.warn(msg) May 17 19:29:36.507907 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: May 17 19:29:36.508625 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdf72bf-06cc-4e11-89b7-a7449c448261 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:36.508832 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17b12c6-f21f-4721-bec6-cf2fa8d0cdb6 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:36.521330 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 780/1568] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 17 19:29:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:29:36.521951 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdf72bf-06cc-4e11-89b7-a7449c448261 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:36.538440 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdf72bf-06cc-4e11-89b7-a7449c448261 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:36.542914 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76205699-a8e3-4fd2-8bee-3dffac2a6769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:36.543813 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76205699-a8e3-4fd2-8bee-3dffac2a6769 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:36.544093 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76205699-a8e3-4fd2-8bee-3dffac2a6769 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:36.544370 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76205699-a8e3-4fd2-8bee-3dffac2a6769 admin admin] PATH_INFO: `/v3/projects/7fec6239168e4f15946ccff5cd48d145/users/61876ef673e14d8f849099c9972c9e2f/roles/114e6a16e63447d1924707ace5aace25` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:36.558601 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76205699-a8e3-4fd2-8bee-3dffac2a6769 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fec6239168e4f15946ccff5cd48d145, user_id=61876ef673e14d8f849099c9972c9e2f, role_id=114e6a16e63447d1924707ace5aace25)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:36.566307 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76205699-a8e3-4fd2-8bee-3dffac2a6769 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:36.588937 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 789/1569] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 17 19:29:36 2022] POST /identity/v3/projects => generated 381 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:29:36.589913 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17b12c6-f21f-4721-bec6-cf2fa8d0cdb6 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:36.641498 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 781/1570] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:29:36 2022] PUT /identity/v3/projects/7fec6239168e4f15946ccff5cd48d145/users/61876ef673e14d8f849099c9972c9e2f/roles/114e6a16e63447d1924707ace5aace25 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:29:36.642109 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76205699-a8e3-4fd2-8bee-3dffac2a6769 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:36.648905 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17b12c6-f21f-4721-bec6-cf2fa8d0cdb6 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:36.699716 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76205699-a8e3-4fd2-8bee-3dffac2a6769 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:36.702194 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77102c8-0088-4a71-a5dc-e29a1081cedf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:36.703414 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77102c8-0088-4a71-a5dc-e29a1081cedf admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:36.703721 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77102c8-0088-4a71-a5dc-e29a1081cedf admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:36.703991 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77102c8-0088-4a71-a5dc-e29a1081cedf admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:36.704669 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77102c8-0088-4a71-a5dc-e29a1081cedf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:36.704868 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b5a09b-e99e-49ac-b3db-99cb1b4b0834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:36.705509 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b5a09b-e99e-49ac-b3db-99cb1b4b0834 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:36.705783 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b5a09b-e99e-49ac-b3db-99cb1b4b0834 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:36.706048 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b5a09b-e99e-49ac-b3db-99cb1b4b0834 admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:36.706593 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77102c8-0088-4a71-a5dc-e29a1081cedf admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:36.706705 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b5a09b-e99e-49ac-b3db-99cb1b4b0834 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:36.708642 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b5a09b-e99e-49ac-b3db-99cb1b4b0834 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:36.810798 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 790/1571] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 17 19:29:36 2022] POST /identity/v3/projects => generated 389 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:29:36.810798 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77102c8-0088-4a71-a5dc-e29a1081cedf admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:36.823273 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 782/1572] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 17 19:29:36 2022] POST /identity/v3/projects => generated 403 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:29:36.823273 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b5a09b-e99e-49ac-b3db-99cb1b4b0834 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:36.867968 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77102c8-0088-4a71-a5dc-e29a1081cedf admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:36.879688 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b5a09b-e99e-49ac-b3db-99cb1b4b0834 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:36.884170 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440921d7-cfa4-4797-ad64-9d2eb51fd87d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:36.884889 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440921d7-cfa4-4797-ad64-9d2eb51fd87d admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:36.885194 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440921d7-cfa4-4797-ad64-9d2eb51fd87d admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:36.885460 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440921d7-cfa4-4797-ad64-9d2eb51fd87d admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:36.886172 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440921d7-cfa4-4797-ad64-9d2eb51fd87d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:36.887777 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440921d7-cfa4-4797-ad64-9d2eb51fd87d admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:36.898037 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 783/1573] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:29:36 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:29:36.899008 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440921d7-cfa4-4797-ad64-9d2eb51fd87d admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:36.908267 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440921d7-cfa4-4797-ad64-9d2eb51fd87d admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:36.912771 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2445fe-1f37-4a93-a2da-76210b3af32f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:36.913389 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2445fe-1f37-4a93-a2da-76210b3af32f admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:36.913684 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2445fe-1f37-4a93-a2da-76210b3af32f admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:36.913948 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2445fe-1f37-4a93-a2da-76210b3af32f admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:36.914609 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2445fe-1f37-4a93-a2da-76210b3af32f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:36.916646 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2445fe-1f37-4a93-a2da-76210b3af32f admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:36.924446 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f7dd41-616e-4061-9507-62534e02650e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:36.925134 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f7dd41-616e-4061-9507-62534e02650e admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:36.925411 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f7dd41-616e-4061-9507-62534e02650e admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:36.925824 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f7dd41-616e-4061-9507-62534e02650e admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:36.926537 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f7dd41-616e-4061-9507-62534e02650e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:29:36.926841 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f7dd41-616e-4061-9507-62534e02650e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:36.928472 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f7dd41-616e-4061-9507-62534e02650e admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:36.939976 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 791/1574] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 17 19:29:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:29:36.940875 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f7dd41-616e-4061-9507-62534e02650e admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:36.950298 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f7dd41-616e-4061-9507-62534e02650e admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:36.954948 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75709933-8a69-4605-9ac7-a6d35393cd0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:36.955615 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75709933-8a69-4605-9ac7-a6d35393cd0a admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:36.955893 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75709933-8a69-4605-9ac7-a6d35393cd0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:36.956167 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75709933-8a69-4605-9ac7-a6d35393cd0a admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:36.956838 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75709933-8a69-4605-9ac7-a6d35393cd0a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:36.958749 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75709933-8a69-4605-9ac7-a6d35393cd0a admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:37.077343 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 784/1575] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 17 19:29:36 2022] POST /identity/v3/projects => generated 393 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:29:37.079070 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2445fe-1f37-4a93-a2da-76210b3af32f admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:37.100100 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 792/1576] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 17 19:29:36 2022] POST /identity/v3/projects => generated 373 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:29:37.101370 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8152116f-59bc-4ea9-bd73-a49d29806a93 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:37.101654 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8152116f-59bc-4ea9-bd73-a49d29806a93 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:37.101935 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8152116f-59bc-4ea9-bd73-a49d29806a93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:37.128353 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-8152116f-59bc-4ea9-bd73-a49d29806a93 None None] MFA Rules not processed for user `61876ef673e14d8f849099c9972c9e2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:29:37.129597 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-8152116f-59bc-4ea9-bd73-a49d29806a93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:29:37.138434 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2445fe-1f37-4a93-a2da-76210b3af32f admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:37.190495 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 793/1577] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:29:37 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:29:37.191311 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8152116f-59bc-4ea9-bd73-a49d29806a93 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:37.195564 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bf140c-2f38-4ca5-9614-8375a2f4207e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:37.196233 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bf140c-2f38-4ca5-9614-8375a2f4207e admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:37.196506 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bf140c-2f38-4ca5-9614-8375a2f4207e admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:37.197061 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bf140c-2f38-4ca5-9614-8375a2f4207e admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:37.197785 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bf140c-2f38-4ca5-9614-8375a2f4207e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:37.199962 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bf140c-2f38-4ca5-9614-8375a2f4207e admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:37.205200 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8152116f-59bc-4ea9-bd73-a49d29806a93 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:37.210208 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41f30de-3d97-41db-bebc-5db3c0ed41b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:37.210862 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41f30de-3d97-41db-bebc-5db3c0ed41b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:37.211140 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41f30de-3d97-41db-bebc-5db3c0ed41b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:37.211471 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41f30de-3d97-41db-bebc-5db3c0ed41b4 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:37.212164 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41f30de-3d97-41db-bebc-5db3c0ed41b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:37.214087 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41f30de-3d97-41db-bebc-5db3c0ed41b4 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:37.309155 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 785/1578] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 17 19:29:37 2022] POST /identity/v3/users => generated 291 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:29:37.309951 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bf140c-2f38-4ca5-9614-8375a2f4207e admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:37.319291 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bf140c-2f38-4ca5-9614-8375a2f4207e admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:37.323924 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f04d61a-f8c8-4121-af65-6dd0dba35c02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:37.324587 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f04d61a-f8c8-4121-af65-6dd0dba35c02 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:37.324866 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f04d61a-f8c8-4121-af65-6dd0dba35c02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:37.325134 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f04d61a-f8c8-4121-af65-6dd0dba35c02 admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:37.325808 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f04d61a-f8c8-4121-af65-6dd0dba35c02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:37.328283 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f04d61a-f8c8-4121-af65-6dd0dba35c02 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:37.340704 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 794/1579] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 17 19:29:37 2022] POST /identity/v3/users => generated 295 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:29:37.340704 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41f30de-3d97-41db-bebc-5db3c0ed41b4 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:37.348547 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41f30de-3d97-41db-bebc-5db3c0ed41b4 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:37.352980 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0d4179-12c5-442a-be3c-469fbda60501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:37.353988 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0d4179-12c5-442a-be3c-469fbda60501 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:37.353988 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0d4179-12c5-442a-be3c-469fbda60501 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:37.355375 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0d4179-12c5-442a-be3c-469fbda60501 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:37.355375 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0d4179-12c5-442a-be3c-469fbda60501 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:37.356412 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0d4179-12c5-442a-be3c-469fbda60501 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:37.366104 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 795/1580] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:29:37 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:29:37.366860 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0d4179-12c5-442a-be3c-469fbda60501 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:37.375736 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0d4179-12c5-442a-be3c-469fbda60501 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:37.380198 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26de7112-9c78-42dd-9e91-dcc333ad8370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:37.383537 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26de7112-9c78-42dd-9e91-dcc333ad8370 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:37.383537 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26de7112-9c78-42dd-9e91-dcc333ad8370 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:37.383537 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26de7112-9c78-42dd-9e91-dcc333ad8370 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:37.383537 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26de7112-9c78-42dd-9e91-dcc333ad8370 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:37.383887 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26de7112-9c78-42dd-9e91-dcc333ad8370 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:37.401917 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 786/1581] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 17 19:29:37 2022] POST /identity/v3/users => generated 302 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:29:37.401917 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f04d61a-f8c8-4121-af65-6dd0dba35c02 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:37.411738 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f04d61a-f8c8-4121-af65-6dd0dba35c02 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:37.416276 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df9d0ba-2545-449d-b225-c5d7556b0b73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:37.416949 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df9d0ba-2545-449d-b225-c5d7556b0b73 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:37.417238 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df9d0ba-2545-449d-b225-c5d7556b0b73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:37.417491 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df9d0ba-2545-449d-b225-c5d7556b0b73 admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:37.418155 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df9d0ba-2545-449d-b225-c5d7556b0b73 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:37.420140 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df9d0ba-2545-449d-b225-c5d7556b0b73 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:37.481935 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 796/1582] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 17 19:29:37 2022] POST /identity/v3/projects => generated 369 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:29:37.482734 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26de7112-9c78-42dd-9e91-dcc333ad8370 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:37.532553 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 787/1583] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 17 19:29:37 2022] POST /identity/v3/users => generated 297 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:29:37.533878 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05f1cf3-6a8e-420b-8c56-d4ef1e01b81b None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:37.534164 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05f1cf3-6a8e-420b-8c56-d4ef1e01b81b None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:37.534463 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05f1cf3-6a8e-420b-8c56-d4ef1e01b81b None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:37.536100 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 788/1584] 10.222.0.58 () {54 vars in 961 bytes} [Tue May 17 19:29:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:29:37.539207 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05f1cf3-6a8e-420b-8c56-d4ef1e01b81b None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:37.541928 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26de7112-9c78-42dd-9e91-dcc333ad8370 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:37.546633 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f401df-4cc1-418a-9fa6-fcdb47536f27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:37.547427 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f401df-4cc1-418a-9fa6-fcdb47536f27 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:37.547813 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f401df-4cc1-418a-9fa6-fcdb47536f27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:37.548089 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f401df-4cc1-418a-9fa6-fcdb47536f27 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:37.548767 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f401df-4cc1-418a-9fa6-fcdb47536f27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:37.549398 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05f1cf3-6a8e-420b-8c56-d4ef1e01b81b None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:37.550727 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f401df-4cc1-418a-9fa6-fcdb47536f27 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:37.553916 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3b9b63-6ab1-4e6b-868e-20ad298208b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:37.554544 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3b9b63-6ab1-4e6b-868e-20ad298208b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:37.554824 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3b9b63-6ab1-4e6b-868e-20ad298208b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:37.555101 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3b9b63-6ab1-4e6b-868e-20ad298208b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:37.555852 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3b9b63-6ab1-4e6b-868e-20ad298208b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:37.557469 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3b9b63-6ab1-4e6b-868e-20ad298208b9 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:37.568442 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 789/1585] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:29:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:29:37.569225 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3b9b63-6ab1-4e6b-868e-20ad298208b9 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:37.578993 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3b9b63-6ab1-4e6b-868e-20ad298208b9 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:37.583429 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1b64c5-2afa-420a-8052-42244e72b238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:37.584068 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1b64c5-2afa-420a-8052-42244e72b238 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:37.584345 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1b64c5-2afa-420a-8052-42244e72b238 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:37.584642 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1b64c5-2afa-420a-8052-42244e72b238 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:37.585335 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1b64c5-2afa-420a-8052-42244e72b238 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:37.586875 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1b64c5-2afa-420a-8052-42244e72b238 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:37.596244 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 790/1586] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:29:37 2022] GET /identity/v3/roles => generated 1420 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:29:37.597776 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1b64c5-2afa-420a-8052-42244e72b238 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:37.606226 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1b64c5-2afa-420a-8052-42244e72b238 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:37.610666 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69efc8cd-a85d-4bef-bbd3-8c6b16060d4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:37.611470 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69efc8cd-a85d-4bef-bbd3-8c6b16060d4d admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:37.611856 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69efc8cd-a85d-4bef-bbd3-8c6b16060d4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:37.612128 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69efc8cd-a85d-4bef-bbd3-8c6b16060d4d admin admin] PATH_INFO: `/v3/projects/9201351f65334032be89ac50d14926ba/users/fc9129cb9f874b32b99c78349dd58477/roles/114e6a16e63447d1924707ace5aace25` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:37.622626 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 797/1587] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 17 19:29:37 2022] POST /identity/v3/users => generated 287 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:29:37.623557 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f401df-4cc1-418a-9fa6-fcdb47536f27 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:37.628024 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69efc8cd-a85d-4bef-bbd3-8c6b16060d4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9201351f65334032be89ac50d14926ba, user_id=fc9129cb9f874b32b99c78349dd58477, role_id=114e6a16e63447d1924707ace5aace25)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:37.630725 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69efc8cd-a85d-4bef-bbd3-8c6b16060d4d admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:37.633221 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f401df-4cc1-418a-9fa6-fcdb47536f27 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:37.638668 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80981ff2-bc1a-47bd-8b74-402d44ec70b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:37.638668 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80981ff2-bc1a-47bd-8b74-402d44ec70b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:37.638668 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80981ff2-bc1a-47bd-8b74-402d44ec70b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:37.638972 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80981ff2-bc1a-47bd-8b74-402d44ec70b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:37.639634 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80981ff2-bc1a-47bd-8b74-402d44ec70b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:37.641153 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80981ff2-bc1a-47bd-8b74-402d44ec70b2 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:37.650633 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 798/1588] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:29:37 2022] GET /identity/v3/roles => generated 1420 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:29:37.651366 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80981ff2-bc1a-47bd-8b74-402d44ec70b2 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:37.660762 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80981ff2-bc1a-47bd-8b74-402d44ec70b2 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:37.665383 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d32ea29-22b3-4929-b3ed-88edbcb46500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:37.668686 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d32ea29-22b3-4929-b3ed-88edbcb46500 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:37.668686 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d32ea29-22b3-4929-b3ed-88edbcb46500 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:37.668686 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d32ea29-22b3-4929-b3ed-88edbcb46500 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:37.668686 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d32ea29-22b3-4929-b3ed-88edbcb46500 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:37.669160 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d32ea29-22b3-4929-b3ed-88edbcb46500 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:37.721951 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 791/1589] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:29:37 2022] PUT /identity/v3/projects/9201351f65334032be89ac50d14926ba/users/fc9129cb9f874b32b99c78349dd58477/roles/114e6a16e63447d1924707ace5aace25 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:29:37.722924 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69efc8cd-a85d-4bef-bbd3-8c6b16060d4d admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:37.749637 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 799/1590] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 17 19:29:37 2022] POST /identity/v3/users => generated 285 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:29:37.754843 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412d1d3f-25e9-4099-b3d3-e2158cc27e19 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:37.754843 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412d1d3f-25e9-4099-b3d3-e2158cc27e19 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:37.754843 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412d1d3f-25e9-4099-b3d3-e2158cc27e19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:37.770748 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-412d1d3f-25e9-4099-b3d3-e2158cc27e19 None None] MFA Rules not processed for user `65c1f878b0dc4d63a4f5dcb12fac1dc1`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:29:37.772064 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-412d1d3f-25e9-4099-b3d3-e2158cc27e19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:29:37.783522 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69efc8cd-a85d-4bef-bbd3-8c6b16060d4d admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:37.788687 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbb83fd-2a2a-44de-8d9b-fb67170d3d2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:37.789497 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbb83fd-2a2a-44de-8d9b-fb67170d3d2d admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:37.789934 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbb83fd-2a2a-44de-8d9b-fb67170d3d2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:37.790456 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbb83fd-2a2a-44de-8d9b-fb67170d3d2d admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:37.791264 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbb83fd-2a2a-44de-8d9b-fb67170d3d2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:37.793238 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbb83fd-2a2a-44de-8d9b-fb67170d3d2d admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:37.803585 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 792/1591] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:29:37 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:29:37.804372 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbb83fd-2a2a-44de-8d9b-fb67170d3d2d admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:37.813514 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbb83fd-2a2a-44de-8d9b-fb67170d3d2d admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:37.818060 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dc1061-cb43-4111-9db6-4fd0e28a21a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:37.818676 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dc1061-cb43-4111-9db6-4fd0e28a21a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:37.818953 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dc1061-cb43-4111-9db6-4fd0e28a21a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:37.819220 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dc1061-cb43-4111-9db6-4fd0e28a21a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:37.819943 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dc1061-cb43-4111-9db6-4fd0e28a21a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:37.821506 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dc1061-cb43-4111-9db6-4fd0e28a21a7 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:37.831566 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 793/1592] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:29:37 2022] GET /identity/v3/roles => generated 1420 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:29:37.832273 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dc1061-cb43-4111-9db6-4fd0e28a21a7 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:37.839369 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 800/1593] 10.222.0.58 () {58 vars in 1058 bytes} [Tue May 17 19:29:37 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:29:37.840443 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412d1d3f-25e9-4099-b3d3-e2158cc27e19 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:37.841194 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dc1061-cb43-4111-9db6-4fd0e28a21a7 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:37.846233 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519c3cae-bdfd-41e8-a20b-cf67ae3984c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:37.847113 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519c3cae-bdfd-41e8-a20b-cf67ae3984c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:37.847859 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519c3cae-bdfd-41e8-a20b-cf67ae3984c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:37.847859 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519c3cae-bdfd-41e8-a20b-cf67ae3984c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:37.848431 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519c3cae-bdfd-41e8-a20b-cf67ae3984c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:37.849977 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519c3cae-bdfd-41e8-a20b-cf67ae3984c8 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:37.851430 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412d1d3f-25e9-4099-b3d3-e2158cc27e19 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:37.856027 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a370bacc-581d-46a3-ac58-08202dbe4ecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:37.856675 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a370bacc-581d-46a3-ac58-08202dbe4ecb admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:37.856959 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a370bacc-581d-46a3-ac58-08202dbe4ecb admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:37.857230 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a370bacc-581d-46a3-ac58-08202dbe4ecb admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:37.858101 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a370bacc-581d-46a3-ac58-08202dbe4ecb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:37.859745 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a370bacc-581d-46a3-ac58-08202dbe4ecb admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:37.860629 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 794/1594] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:29:37 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:29:37.861430 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519c3cae-bdfd-41e8-a20b-cf67ae3984c8 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:37.869769 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 801/1595] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:29:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:29:37.871013 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5e784e-feeb-460b-add9-d2f7581704b9 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:37.874991 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5e784e-feeb-460b-add9-d2f7581704b9 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:37.874991 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5e784e-feeb-460b-add9-d2f7581704b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:37.875192 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519c3cae-bdfd-41e8-a20b-cf67ae3984c8 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:37.878897 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440ff732-7aa1-4c27-94cb-bd979aee4059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:37.879575 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440ff732-7aa1-4c27-94cb-bd979aee4059 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:37.879856 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440ff732-7aa1-4c27-94cb-bd979aee4059 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:37.880130 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440ff732-7aa1-4c27-94cb-bd979aee4059 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:37.880837 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440ff732-7aa1-4c27-94cb-bd979aee4059 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:37.882453 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440ff732-7aa1-4c27-94cb-bd979aee4059 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:37.888394 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-cf5e784e-feeb-460b-add9-d2f7581704b9 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:29:37.889569 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-cf5e784e-feeb-460b-add9-d2f7581704b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:29:37.895769 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 795/1596] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:29:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:29:37.895769 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440ff732-7aa1-4c27-94cb-bd979aee4059 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:37.899261 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 802/1597] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:29:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:29:37.900073 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5e784e-feeb-460b-add9-d2f7581704b9 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:37.902904 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440ff732-7aa1-4c27-94cb-bd979aee4059 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:37.908722 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1cbddb-6c05-496c-8057-5a9c4f5402e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:37.909237 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5e784e-feeb-460b-add9-d2f7581704b9 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:37.909362 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1cbddb-6c05-496c-8057-5a9c4f5402e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:37.909637 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1cbddb-6c05-496c-8057-5a9c4f5402e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:37.909913 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1cbddb-6c05-496c-8057-5a9c4f5402e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:37.910625 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1cbddb-6c05-496c-8057-5a9c4f5402e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:37.912366 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1cbddb-6c05-496c-8057-5a9c4f5402e7 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:37.913766 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0ba703-2d4e-421b-a367-9ea6b8b2159b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:37.914467 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0ba703-2d4e-421b-a367-9ea6b8b2159b admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:37.914749 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0ba703-2d4e-421b-a367-9ea6b8b2159b admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:37.915141 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0ba703-2d4e-421b-a367-9ea6b8b2159b admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:37.915899 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0ba703-2d4e-421b-a367-9ea6b8b2159b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:37.917487 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0ba703-2d4e-421b-a367-9ea6b8b2159b admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:37.923202 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 796/1598] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:29:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:29:37.923869 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1cbddb-6c05-496c-8057-5a9c4f5402e7 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:37.928292 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 803/1599] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:29:37 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:29:37.929218 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0ba703-2d4e-421b-a367-9ea6b8b2159b admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:37.933147 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1cbddb-6c05-496c-8057-5a9c4f5402e7 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:37.937824 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280fd01a-a295-4f30-ae7a-63a7202c30ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:37.938672 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280fd01a-a295-4f30-ae7a-63a7202c30ab admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:37.938949 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280fd01a-a295-4f30-ae7a-63a7202c30ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:37.939217 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280fd01a-a295-4f30-ae7a-63a7202c30ab admin admin] PATH_INFO: `/v3/projects/c3209fe3cf6b4daaba671bc658f8b1a6/users/33b957d0521c4fcd98f844020b7becd1/roles/114e6a16e63447d1924707ace5aace25` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:37.942516 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0ba703-2d4e-421b-a367-9ea6b8b2159b admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:37.950912 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab29ce3c-6366-421f-b384-aa9375cc01eb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c1f878b0dc4d63a4f5dcb12fac1dc1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:37.951573 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab29ce3c-6366-421f-b384-aa9375cc01eb service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:37.951857 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab29ce3c-6366-421f-b384-aa9375cc01eb service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:37.952150 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab29ce3c-6366-421f-b384-aa9375cc01eb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:37.954050 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280fd01a-a295-4f30-ae7a-63a7202c30ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3209fe3cf6b4daaba671bc658f8b1a6, user_id=33b957d0521c4fcd98f844020b7becd1, role_id=114e6a16e63447d1924707ace5aace25)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:37.956767 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280fd01a-a295-4f30-ae7a-63a7202c30ab admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:38.006051 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab29ce3c-6366-421f-b384-aa9375cc01eb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:38.007676 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab29ce3c-6366-421f-b384-aa9375cc01eb service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:38.016325 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 804/1600] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:29:37 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:29:38.017054 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab29ce3c-6366-421f-b384-aa9375cc01eb service nova] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:38.026114 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab29ce3c-6366-421f-b384-aa9375cc01eb service nova] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:38.030612 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d523d5-5bb2-4430-8c98-4ab5d3539940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:38.031531 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d523d5-5bb2-4430-8c98-4ab5d3539940 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:38.031814 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d523d5-5bb2-4430-8c98-4ab5d3539940 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:38.032086 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d523d5-5bb2-4430-8c98-4ab5d3539940 admin admin] PATH_INFO: `/v3/projects/b23e9f1907a6406785be000e1373dc41/users/6545bb5d5eb24c199b5d89091b2c1545/roles/114e6a16e63447d1924707ace5aace25` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:38.045877 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d523d5-5bb2-4430-8c98-4ab5d3539940 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b23e9f1907a6406785be000e1373dc41, user_id=6545bb5d5eb24c199b5d89091b2c1545, role_id=114e6a16e63447d1924707ace5aace25)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:38.047412 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: WARNING py.warnings [None req-35d523d5-5bb2-4430-8c98-4ab5d3539940 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 17 19:29:38.047412 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: warnings.warn(msg) May 17 19:29:38.047412 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: May 17 19:29:38.049436 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d523d5-5bb2-4430-8c98-4ab5d3539940 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:38.053948 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 797/1601] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:29:37 2022] PUT /identity/v3/projects/c3209fe3cf6b4daaba671bc658f8b1a6/users/33b957d0521c4fcd98f844020b7becd1/roles/114e6a16e63447d1924707ace5aace25 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:29:38.054790 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280fd01a-a295-4f30-ae7a-63a7202c30ab admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:38.113886 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280fd01a-a295-4f30-ae7a-63a7202c30ab admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:38.118267 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23c1607-50e9-4d56-9e5a-3c86e07ebe76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:38.118941 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23c1607-50e9-4d56-9e5a-3c86e07ebe76 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:38.119214 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23c1607-50e9-4d56-9e5a-3c86e07ebe76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:38.119506 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23c1607-50e9-4d56-9e5a-3c86e07ebe76 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:38.120215 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23c1607-50e9-4d56-9e5a-3c86e07ebe76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:38.121795 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23c1607-50e9-4d56-9e5a-3c86e07ebe76 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:38.132659 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 798/1602] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:29:38 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:29:38.136476 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23c1607-50e9-4d56-9e5a-3c86e07ebe76 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:38.136940 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 805/1603] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:29:38 2022] PUT /identity/v3/projects/b23e9f1907a6406785be000e1373dc41/users/6545bb5d5eb24c199b5d89091b2c1545/roles/114e6a16e63447d1924707ace5aace25 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:29:38.137947 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d523d5-5bb2-4430-8c98-4ab5d3539940 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:38.146691 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23c1607-50e9-4d56-9e5a-3c86e07ebe76 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:38.192942 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d523d5-5bb2-4430-8c98-4ab5d3539940 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:38.197009 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2568e7a9-8f17-46d4-9513-e5318a00cd19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:38.197709 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2568e7a9-8f17-46d4-9513-e5318a00cd19 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:38.198055 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2568e7a9-8f17-46d4-9513-e5318a00cd19 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:38.198392 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2568e7a9-8f17-46d4-9513-e5318a00cd19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:38.205328 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a647fbb1-6b08-4d8f-8d80-269577101811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:38.206208 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a647fbb1-6b08-4d8f-8d80-269577101811 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:38.206486 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a647fbb1-6b08-4d8f-8d80-269577101811 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:38.207028 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a647fbb1-6b08-4d8f-8d80-269577101811 admin admin] PATH_INFO: `/v3/projects/4425ca36d0fd4d779229d4003a0a607c/users/b652b831bbd24934a21d7aa4c8dfeee7/roles/114e6a16e63447d1924707ace5aace25` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:38.221362 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a647fbb1-6b08-4d8f-8d80-269577101811 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4425ca36d0fd4d779229d4003a0a607c, user_id=b652b831bbd24934a21d7aa4c8dfeee7, role_id=114e6a16e63447d1924707ace5aace25)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:38.224330 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a647fbb1-6b08-4d8f-8d80-269577101811 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:38.258356 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2568e7a9-8f17-46d4-9513-e5318a00cd19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:38.259898 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2568e7a9-8f17-46d4-9513-e5318a00cd19 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:38.270055 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 806/1604] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:29:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:29:38.270841 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2568e7a9-8f17-46d4-9513-e5318a00cd19 service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:38.279399 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2568e7a9-8f17-46d4-9513-e5318a00cd19 service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:38.284344 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc99599-c269-4e8d-90ea-3f8c79f49f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:38.284989 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc99599-c269-4e8d-90ea-3f8c79f49f7f admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:38.285311 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc99599-c269-4e8d-90ea-3f8c79f49f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:38.285536 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc99599-c269-4e8d-90ea-3f8c79f49f7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:38.286246 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc99599-c269-4e8d-90ea-3f8c79f49f7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:38.287879 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc99599-c269-4e8d-90ea-3f8c79f49f7f admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:38.298164 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 799/1605] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:29:38 2022] PUT /identity/v3/projects/4425ca36d0fd4d779229d4003a0a607c/users/b652b831bbd24934a21d7aa4c8dfeee7/roles/114e6a16e63447d1924707ace5aace25 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:29:38.298962 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a647fbb1-6b08-4d8f-8d80-269577101811 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:38.302259 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 807/1606] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:29:38 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:29:38.302259 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989b3f69-39a7-4e7a-88b0-5646b8ecdf8b None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:38.302259 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989b3f69-39a7-4e7a-88b0-5646b8ecdf8b None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:38.302259 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989b3f69-39a7-4e7a-88b0-5646b8ecdf8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:38.326790 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-989b3f69-39a7-4e7a-88b0-5646b8ecdf8b None None] MFA Rules not processed for user `33b957d0521c4fcd98f844020b7becd1`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:29:38.328108 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-989b3f69-39a7-4e7a-88b0-5646b8ecdf8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:29:38.361187 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a647fbb1-6b08-4d8f-8d80-269577101811 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:38.365810 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a276dd9c-c872-4163-a7c4-88b2ca7c25ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:38.366710 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a276dd9c-c872-4163-a7c4-88b2ca7c25ef admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:38.366969 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a276dd9c-c872-4163-a7c4-88b2ca7c25ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:38.367239 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a276dd9c-c872-4163-a7c4-88b2ca7c25ef admin admin] PATH_INFO: `/v3/projects/5d488b25563d4373b5c980ab060bc822/users/b171cf633e6e437fb509857b0a2962e2/roles/114e6a16e63447d1924707ace5aace25` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:38.383215 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a276dd9c-c872-4163-a7c4-88b2ca7c25ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d488b25563d4373b5c980ab060bc822, user_id=b171cf633e6e437fb509857b0a2962e2, role_id=114e6a16e63447d1924707ace5aace25)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:38.386450 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a276dd9c-c872-4163-a7c4-88b2ca7c25ef admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:38.393553 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 808/1607] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:29:38 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:29:38.394482 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989b3f69-39a7-4e7a-88b0-5646b8ecdf8b None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:38.404448 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989b3f69-39a7-4e7a-88b0-5646b8ecdf8b None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:38.409050 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01630605-c72d-4cea-a007-d3fbe0e62421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:38.409916 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01630605-c72d-4cea-a007-d3fbe0e62421 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:38.410186 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01630605-c72d-4cea-a007-d3fbe0e62421 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:38.410451 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01630605-c72d-4cea-a007-d3fbe0e62421 admin admin] PATH_INFO: `/v3/projects/947298bea89442fea518ea0a65c1e7cb/users/6f6b0266e7914eaf9f1103addc78c485/roles/114e6a16e63447d1924707ace5aace25` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:38.424581 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01630605-c72d-4cea-a007-d3fbe0e62421 admin admin] RBAC: Authorizing `identity:create_grant(project_id=947298bea89442fea518ea0a65c1e7cb, user_id=6f6b0266e7914eaf9f1103addc78c485, role_id=114e6a16e63447d1924707ace5aace25)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:38.427383 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01630605-c72d-4cea-a007-d3fbe0e62421 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:38.496335 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 800/1608] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:29:38 2022] PUT /identity/v3/projects/5d488b25563d4373b5c980ab060bc822/users/b171cf633e6e437fb509857b0a2962e2/roles/114e6a16e63447d1924707ace5aace25 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:29:38.497762 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9e8acc-518d-46ad-b925-4ff70ac8763c None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:38.500041 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9e8acc-518d-46ad-b925-4ff70ac8763c None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:38.500041 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9e8acc-518d-46ad-b925-4ff70ac8763c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:38.514630 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-ce9e8acc-518d-46ad-b925-4ff70ac8763c None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:29:38.516427 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-ce9e8acc-518d-46ad-b925-4ff70ac8763c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:29:38.525340 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 809/1609] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:29:38 2022] PUT /identity/v3/projects/947298bea89442fea518ea0a65c1e7cb/users/6f6b0266e7914eaf9f1103addc78c485/roles/114e6a16e63447d1924707ace5aace25 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:29:38.536125 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831d7465-037b-455d-8181-1c98fef6035f None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:38.536488 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831d7465-037b-455d-8181-1c98fef6035f None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:38.536745 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831d7465-037b-455d-8181-1c98fef6035f None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:38.538285 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 810/1610] 10.222.0.58 () {54 vars in 912 bytes} [Tue May 17 19:29:38 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:29:38.539199 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422a00f4-244b-41a8-9037-b6c486b349f2 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:38.539479 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422a00f4-244b-41a8-9037-b6c486b349f2 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:38.539772 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422a00f4-244b-41a8-9037-b6c486b349f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:38.558560 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-422a00f4-244b-41a8-9037-b6c486b349f2 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:29:38.559993 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-422a00f4-244b-41a8-9037-b6c486b349f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:29:38.580757 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 801/1611] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:29:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:29:38.580757 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9e8acc-518d-46ad-b925-4ff70ac8763c None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:38.588629 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9e8acc-518d-46ad-b925-4ff70ac8763c None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:38.639676 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 811/1612] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:29:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:29:38.639676 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422a00f4-244b-41a8-9037-b6c486b349f2 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:38.648206 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422a00f4-244b-41a8-9037-b6c486b349f2 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:38.651878 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47de54fe-acc3-4d65-8f55-0384ed6e809f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:38.652772 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47de54fe-acc3-4d65-8f55-0384ed6e809f admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:38.657996 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47de54fe-acc3-4d65-8f55-0384ed6e809f admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:38.658118 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea03eab8-9c36-4c2c-b17b-caf193a67959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:38.658234 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47de54fe-acc3-4d65-8f55-0384ed6e809f admin admin] PATH_INFO: `/v3/projects/a1b94ef50c204b5e95f741ded1bcc77f/users/5ce69eb7e8cf40539ef9dc09d8a22f38/roles/114e6a16e63447d1924707ace5aace25` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:38.658327 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea03eab8-9c36-4c2c-b17b-caf193a67959 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:38.658327 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea03eab8-9c36-4c2c-b17b-caf193a67959 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:38.658327 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea03eab8-9c36-4c2c-b17b-caf193a67959 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:38.658327 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea03eab8-9c36-4c2c-b17b-caf193a67959 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:38.658327 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea03eab8-9c36-4c2c-b17b-caf193a67959 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:38.667377 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47de54fe-acc3-4d65-8f55-0384ed6e809f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1b94ef50c204b5e95f741ded1bcc77f, user_id=5ce69eb7e8cf40539ef9dc09d8a22f38, role_id=114e6a16e63447d1924707ace5aace25)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:38.670057 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47de54fe-acc3-4d65-8f55-0384ed6e809f admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:38.770209 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 802/1613] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:29:38 2022] PUT /identity/v3/projects/a1b94ef50c204b5e95f741ded1bcc77f/users/5ce69eb7e8cf40539ef9dc09d8a22f38/roles/114e6a16e63447d1924707ace5aace25 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:29:38.771514 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1b1d40-1fcb-40c8-bf19-90e34572e84e None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:38.771807 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1b1d40-1fcb-40c8-bf19-90e34572e84e None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:38.772072 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1b1d40-1fcb-40c8-bf19-90e34572e84e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:38.781607 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 812/1614] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 17 19:29:38 2022] POST /identity/v3/projects => generated 379 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:29:38.783272 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bed920-b840-482a-b855-e81e9cd04350 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:38.783651 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bed920-b840-482a-b855-e81e9cd04350 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:38.783996 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bed920-b840-482a-b855-e81e9cd04350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:38.791404 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-9f1b1d40-1fcb-40c8-bf19-90e34572e84e None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:29:38.792666 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-9f1b1d40-1fcb-40c8-bf19-90e34572e84e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:29:38.815910 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-19bed920-b840-482a-b855-e81e9cd04350 None None] MFA Rules not processed for user `6f6b0266e7914eaf9f1103addc78c485`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:29:38.817644 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-19bed920-b840-482a-b855-e81e9cd04350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:29:38.867554 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 803/1615] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:29:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:29:38.869385 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab75d1f2-eaeb-4ea8-a61f-6226b123fbac None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:38.869958 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab75d1f2-eaeb-4ea8-a61f-6226b123fbac None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:38.870543 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab75d1f2-eaeb-4ea8-a61f-6226b123fbac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:38.891924 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-ab75d1f2-eaeb-4ea8-a61f-6226b123fbac None None] MFA Rules not processed for user `65c1f878b0dc4d63a4f5dcb12fac1dc1`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:29:38.893195 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-ab75d1f2-eaeb-4ea8-a61f-6226b123fbac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:29:38.900895 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 813/1616] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:29:38 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:29:38.901894 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bed920-b840-482a-b855-e81e9cd04350 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:38.993515 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bed920-b840-482a-b855-e81e9cd04350 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:38.997250 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d12d35-4f45-4bd3-8100-aa27cbcba3f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:38.997943 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d12d35-4f45-4bd3-8100-aa27cbcba3f3 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:38.998226 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d12d35-4f45-4bd3-8100-aa27cbcba3f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:38.998941 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d12d35-4f45-4bd3-8100-aa27cbcba3f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:39.005994 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 804/1617] 10.222.0.58 () {58 vars in 1009 bytes} [Tue May 17 19:29:38 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:29:39.006853 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab75d1f2-eaeb-4ea8-a61f-6226b123fbac None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:39.008693 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d12d35-4f45-4bd3-8100-aa27cbcba3f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:39.011612 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d12d35-4f45-4bd3-8100-aa27cbcba3f3 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:39.016304 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab75d1f2-eaeb-4ea8-a61f-6226b123fbac None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:39.020111 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79305ca7-a553-4f26-8562-3634615e6319 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:39.021030 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79305ca7-a553-4f26-8562-3634615e6319 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:39.021360 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79305ca7-a553-4f26-8562-3634615e6319 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:39.023146 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79305ca7-a553-4f26-8562-3634615e6319 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:39.023744 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 814/1618] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:29:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:29:39.024909 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8667ce1a-e1ff-4dec-95f6-df3c7233bff5 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:39.025198 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8667ce1a-e1ff-4dec-95f6-df3c7233bff5 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:39.025472 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8667ce1a-e1ff-4dec-95f6-df3c7233bff5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:39.033468 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79305ca7-a553-4f26-8562-3634615e6319 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:39.035018 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79305ca7-a553-4f26-8562-3634615e6319 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:39.045422 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 805/1619] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:29:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:29:39.047099 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79305ca7-a553-4f26-8562-3634615e6319 service neutron] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:39.055760 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-8667ce1a-e1ff-4dec-95f6-df3c7233bff5 None None] MFA Rules not processed for user `5ce69eb7e8cf40539ef9dc09d8a22f38`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:29:39.056599 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79305ca7-a553-4f26-8562-3634615e6319 service neutron] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:39.057095 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-8667ce1a-e1ff-4dec-95f6-df3c7233bff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:29:39.062027 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cab63f8-6c1c-4f15-8948-6a47bf263fbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:39.062796 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cab63f8-6c1c-4f15-8948-6a47bf263fbe admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:39.063222 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cab63f8-6c1c-4f15-8948-6a47bf263fbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:39.063623 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cab63f8-6c1c-4f15-8948-6a47bf263fbe admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:39.071095 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cab63f8-6c1c-4f15-8948-6a47bf263fbe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:39.071095 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cab63f8-6c1c-4f15-8948-6a47bf263fbe admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:39.125181 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 815/1620] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:29:39 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:29:39.126210 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8667ce1a-e1ff-4dec-95f6-df3c7233bff5 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:39.134883 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8667ce1a-e1ff-4dec-95f6-df3c7233bff5 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:39.138434 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828e8013-6a6b-4357-9534-b8c7ca3042fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:39.139091 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828e8013-6a6b-4357-9534-b8c7ca3042fc service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:39.139365 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828e8013-6a6b-4357-9534-b8c7ca3042fc service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:39.139664 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828e8013-6a6b-4357-9534-b8c7ca3042fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:39.150381 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828e8013-6a6b-4357-9534-b8c7ca3042fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:39.153728 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828e8013-6a6b-4357-9534-b8c7ca3042fc service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:39.162172 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 806/1621] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 17 19:29:39 2022] POST /identity/v3/users => generated 296 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:29:39.163040 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 816/1622] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:29:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:29:39.163291 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cab63f8-6c1c-4f15-8948-6a47bf263fbe admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:39.163634 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828e8013-6a6b-4357-9534-b8c7ca3042fc service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:39.173392 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cab63f8-6c1c-4f15-8948-6a47bf263fbe admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:39.178693 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a69b2b1-8e27-463b-a4f6-c9c13663e2f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:39.179672 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a69b2b1-8e27-463b-a4f6-c9c13663e2f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:39.179955 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a69b2b1-8e27-463b-a4f6-c9c13663e2f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:39.180230 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a69b2b1-8e27-463b-a4f6-c9c13663e2f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:39.180915 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a69b2b1-8e27-463b-a4f6-c9c13663e2f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:39.182815 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a69b2b1-8e27-463b-a4f6-c9c13663e2f0 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:39.220066 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828e8013-6a6b-4357-9534-b8c7ca3042fc service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:39.224092 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353471a1-173a-4ef3-baca-a3d85c889153 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd05409b38ae346458abfe41ebda1c59e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:39.224797 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353471a1-173a-4ef3-baca-a3d85c889153 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:39.225143 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353471a1-173a-4ef3-baca-a3d85c889153 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:39.225476 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353471a1-173a-4ef3-baca-a3d85c889153 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:39.236045 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353471a1-173a-4ef3-baca-a3d85c889153 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:39.237502 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353471a1-173a-4ef3-baca-a3d85c889153 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:39.248992 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 817/1623] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:29:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:29:39.249486 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353471a1-173a-4ef3-baca-a3d85c889153 service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:39.259871 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353471a1-173a-4ef3-baca-a3d85c889153 service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:39.264665 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e6975e-197d-4834-94fe-e20ecb96bddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:39.265358 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e6975e-197d-4834-94fe-e20ecb96bddd admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:39.265638 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e6975e-197d-4834-94fe-e20ecb96bddd admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:39.265910 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e6975e-197d-4834-94fe-e20ecb96bddd admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:39.266593 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e6975e-197d-4834-94fe-e20ecb96bddd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:39.268550 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e6975e-197d-4834-94fe-e20ecb96bddd admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:39.282816 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 807/1624] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 17 19:29:39 2022] POST /identity/v3/projects => generated 371 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:29:39.284304 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a69b2b1-8e27-463b-a4f6-c9c13663e2f0 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:39.350553 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a69b2b1-8e27-463b-a4f6-c9c13663e2f0 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:39.355642 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6bf3b4-0b3c-4d53-bdb8-62276ea0c3bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:39.356338 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6bf3b4-0b3c-4d53-bdb8-62276ea0c3bb admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:39.356622 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6bf3b4-0b3c-4d53-bdb8-62276ea0c3bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:39.356899 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6bf3b4-0b3c-4d53-bdb8-62276ea0c3bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:39.357624 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6bf3b4-0b3c-4d53-bdb8-62276ea0c3bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:39.359452 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6bf3b4-0b3c-4d53-bdb8-62276ea0c3bb admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:39.371325 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 808/1625] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:29:39 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:29:39.372367 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6bf3b4-0b3c-4d53-bdb8-62276ea0c3bb admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:39.384334 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6bf3b4-0b3c-4d53-bdb8-62276ea0c3bb admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:39.390419 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2607b3e8-e942-4f88-bf0e-60c21db0c497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:39.391229 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2607b3e8-e942-4f88-bf0e-60c21db0c497 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:39.391658 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2607b3e8-e942-4f88-bf0e-60c21db0c497 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:39.392000 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2607b3e8-e942-4f88-bf0e-60c21db0c497 admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:39.392686 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2607b3e8-e942-4f88-bf0e-60c21db0c497 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:39.397996 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2607b3e8-e942-4f88-bf0e-60c21db0c497 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:39.427050 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 818/1626] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 17 19:29:39 2022] POST /identity/v3/projects => generated 367 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:29:39.427050 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e6975e-197d-4834-94fe-e20ecb96bddd admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:39.505212 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e6975e-197d-4834-94fe-e20ecb96bddd admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:39.517529 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fe4336-e7e1-49cc-8124-c3f5e2eecb0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:39.518912 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fe4336-e7e1-49cc-8124-c3f5e2eecb0f admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:39.519636 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fe4336-e7e1-49cc-8124-c3f5e2eecb0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:39.520292 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fe4336-e7e1-49cc-8124-c3f5e2eecb0f admin admin] PATH_INFO: `/v3/projects/2a3df6eacaef4fc3afc55a204cf6d07a/users/a9d0d1805c2c47549990b42dc1f91500/roles/60af5088793e4379b88821a86552a344` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:39.534520 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 809/1627] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 17 19:29:39 2022] POST /identity/v3/users => generated 292 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:29:39.535863 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2607b3e8-e942-4f88-bf0e-60c21db0c497 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:39.539438 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fe4336-e7e1-49cc-8124-c3f5e2eecb0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a3df6eacaef4fc3afc55a204cf6d07a, user_id=a9d0d1805c2c47549990b42dc1f91500, role_id=60af5088793e4379b88821a86552a344)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:39.542503 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fe4336-e7e1-49cc-8124-c3f5e2eecb0f admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:39.553775 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2607b3e8-e942-4f88-bf0e-60c21db0c497 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:39.559067 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071d895a-7ffe-4692-8766-7c3562b7d2fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:39.559764 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071d895a-7ffe-4692-8766-7c3562b7d2fb admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:39.560046 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071d895a-7ffe-4692-8766-7c3562b7d2fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:39.560319 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071d895a-7ffe-4692-8766-7c3562b7d2fb admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:39.561037 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071d895a-7ffe-4692-8766-7c3562b7d2fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:39.563134 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071d895a-7ffe-4692-8766-7c3562b7d2fb admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:39.667453 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 819/1628] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:29:39 2022] PUT /identity/v3/projects/2a3df6eacaef4fc3afc55a204cf6d07a/users/a9d0d1805c2c47549990b42dc1f91500/roles/60af5088793e4379b88821a86552a344 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:29:39.667453 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fe4336-e7e1-49cc-8124-c3f5e2eecb0f admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:39.700562 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 810/1629] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 17 19:29:39 2022] POST /identity/v3/users => generated 290 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:29:39.703262 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071d895a-7ffe-4692-8766-7c3562b7d2fb admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:39.714476 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071d895a-7ffe-4692-8766-7c3562b7d2fb admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:39.742925 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fe4336-e7e1-49cc-8124-c3f5e2eecb0f admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:39.748728 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea7176e-25fc-4ecc-b774-4bcab64aef7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:39.749417 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea7176e-25fc-4ecc-b774-4bcab64aef7a admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:39.749706 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea7176e-25fc-4ecc-b774-4bcab64aef7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:39.750729 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea7176e-25fc-4ecc-b774-4bcab64aef7a admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:39.751524 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea7176e-25fc-4ecc-b774-4bcab64aef7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:39.753191 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea7176e-25fc-4ecc-b774-4bcab64aef7a admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:39.764781 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 820/1630] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:29:39 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:29:39.765731 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea7176e-25fc-4ecc-b774-4bcab64aef7a admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:39.776348 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea7176e-25fc-4ecc-b774-4bcab64aef7a admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:39.781954 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a29084-e16a-4cbf-b5b4-20f430849f53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:39.793925 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a29084-e16a-4cbf-b5b4-20f430849f53 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:39.793925 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a29084-e16a-4cbf-b5b4-20f430849f53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:39.793925 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a29084-e16a-4cbf-b5b4-20f430849f53 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:39.793925 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a29084-e16a-4cbf-b5b4-20f430849f53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:39.794432 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d884dcfa-7310-4dc3-b465-8ea16bb1a515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:39.794432 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d884dcfa-7310-4dc3-b465-8ea16bb1a515 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:39.794432 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d884dcfa-7310-4dc3-b465-8ea16bb1a515 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:39.794754 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a29084-e16a-4cbf-b5b4-20f430849f53 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:39.794856 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d884dcfa-7310-4dc3-b465-8ea16bb1a515 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:39.794856 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d884dcfa-7310-4dc3-b465-8ea16bb1a515 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:39.794856 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d884dcfa-7310-4dc3-b465-8ea16bb1a515 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:39.801364 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 811/1631] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:29:39 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:29:39.801364 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a29084-e16a-4cbf-b5b4-20f430849f53 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:39.809882 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a29084-e16a-4cbf-b5b4-20f430849f53 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:39.815061 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c141e9c-998a-4b91-b07a-5e3a1ec2e8cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:39.816128 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c141e9c-998a-4b91-b07a-5e3a1ec2e8cb admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:39.816819 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c141e9c-998a-4b91-b07a-5e3a1ec2e8cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:39.817070 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c141e9c-998a-4b91-b07a-5e3a1ec2e8cb admin admin] PATH_INFO: `/v3/projects/47df41203da54e04a5a646125282a9ee/users/dba6ea49afe042658b8741317869b128/roles/60af5088793e4379b88821a86552a344` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:39.831610 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 821/1632] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:29:39 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:29:39.831610 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d884dcfa-7310-4dc3-b465-8ea16bb1a515 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:39.840415 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c141e9c-998a-4b91-b07a-5e3a1ec2e8cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=47df41203da54e04a5a646125282a9ee, user_id=dba6ea49afe042658b8741317869b128, role_id=60af5088793e4379b88821a86552a344)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:39.843959 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c141e9c-998a-4b91-b07a-5e3a1ec2e8cb admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:39.845172 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d884dcfa-7310-4dc3-b465-8ea16bb1a515 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:39.850459 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ea6603-c99e-4e45-8989-f2ea68a8b196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:39.851548 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ea6603-c99e-4e45-8989-f2ea68a8b196 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:39.851925 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ea6603-c99e-4e45-8989-f2ea68a8b196 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:39.852294 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ea6603-c99e-4e45-8989-f2ea68a8b196 admin admin] PATH_INFO: `/v3/projects/2a3df6eacaef4fc3afc55a204cf6d07a/users/a9d0d1805c2c47549990b42dc1f91500/roles/60af5088793e4379b88821a86552a344` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:39.854429 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ea6603-c99e-4e45-8989-f2ea68a8b196 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a3df6eacaef4fc3afc55a204cf6d07a, user_id=a9d0d1805c2c47549990b42dc1f91500, role_id=60af5088793e4379b88821a86552a344)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:39.863744 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ea6603-c99e-4e45-8989-f2ea68a8b196 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:39.869040 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 822/1633] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:29:39 2022] PUT /identity/v3/projects/2a3df6eacaef4fc3afc55a204cf6d07a/users/a9d0d1805c2c47549990b42dc1f91500/roles/60af5088793e4379b88821a86552a344 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:29:39.869831 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ea6603-c99e-4e45-8989-f2ea68a8b196 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:39.947349 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 812/1634] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:29:39 2022] PUT /identity/v3/projects/47df41203da54e04a5a646125282a9ee/users/dba6ea49afe042658b8741317869b128/roles/60af5088793e4379b88821a86552a344 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:29:39.947349 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81213a1-93d2-4b48-b449-091c6d15bb4a None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:39.947349 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81213a1-93d2-4b48-b449-091c6d15bb4a None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:39.947349 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81213a1-93d2-4b48-b449-091c6d15bb4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:39.981336 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ea6603-c99e-4e45-8989-f2ea68a8b196 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:40.039103 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-b81213a1-93d2-4b48-b449-091c6d15bb4a None None] MFA Rules not processed for user `a9d0d1805c2c47549990b42dc1f91500`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:29:40.065938 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-b81213a1-93d2-4b48-b449-091c6d15bb4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:29:40.174601 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a85d0ca-a90a-4fc9-b7f9-5e6f0b93bfde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:40.175550 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a85d0ca-a90a-4fc9-b7f9-5e6f0b93bfde admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:40.175809 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a85d0ca-a90a-4fc9-b7f9-5e6f0b93bfde admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:40.176088 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a85d0ca-a90a-4fc9-b7f9-5e6f0b93bfde admin admin] PATH_INFO: `/v3/projects/a5a5bfa5186f41698644470d9ecc8943/users/a63a1d17dc364eed8bb25ed64745b04c/roles/60af5088793e4379b88821a86552a344` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:40.211700 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a85d0ca-a90a-4fc9-b7f9-5e6f0b93bfde admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5a5bfa5186f41698644470d9ecc8943, user_id=a63a1d17dc364eed8bb25ed64745b04c, role_id=60af5088793e4379b88821a86552a344)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:40.215276 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a85d0ca-a90a-4fc9-b7f9-5e6f0b93bfde admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:40.222158 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 813/1635] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:29:39 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:29:40.223071 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81213a1-93d2-4b48-b449-091c6d15bb4a None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:40.249606 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81213a1-93d2-4b48-b449-091c6d15bb4a None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:40.258742 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1c00da-fe9d-409c-9fdd-276c43afb3e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:40.258742 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1c00da-fe9d-409c-9fdd-276c43afb3e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:40.258742 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1c00da-fe9d-409c-9fdd-276c43afb3e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:40.258742 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1c00da-fe9d-409c-9fdd-276c43afb3e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:40.259335 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1c00da-fe9d-409c-9fdd-276c43afb3e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:40.259472 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1c00da-fe9d-409c-9fdd-276c43afb3e0 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:40.276229 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 814/1636] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:29:40 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:29:40.276229 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597d9ebc-610c-4136-b32f-bce4206dd5b1 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:40.276229 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597d9ebc-610c-4136-b32f-bce4206dd5b1 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:40.276229 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597d9ebc-610c-4136-b32f-bce4206dd5b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:40.329692 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-597d9ebc-610c-4136-b32f-bce4206dd5b1 None None] MFA Rules not processed for user `fc9129cb9f874b32b99c78349dd58477`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:29:40.341215 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 823/1637] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:29:39 2022] PUT /identity/v3/projects/a5a5bfa5186f41698644470d9ecc8943/users/a63a1d17dc364eed8bb25ed64745b04c/roles/60af5088793e4379b88821a86552a344 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:29:40.343671 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a85d0ca-a90a-4fc9-b7f9-5e6f0b93bfde admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:40.343807 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-597d9ebc-610c-4136-b32f-bce4206dd5b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:29:40.459870 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a85d0ca-a90a-4fc9-b7f9-5e6f0b93bfde admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:40.478308 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1219c1d8-dd54-4629-9d7d-8549737dd8db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd05409b38ae346458abfe41ebda1c59e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:40.481031 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1219c1d8-dd54-4629-9d7d-8549737dd8db service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:40.489236 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 815/1638] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:29:40 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:29:40.490270 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597d9ebc-610c-4136-b32f-bce4206dd5b1 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:40.509109 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1219c1d8-dd54-4629-9d7d-8549737dd8db service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:40.512593 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1219c1d8-dd54-4629-9d7d-8549737dd8db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:40.625223 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597d9ebc-610c-4136-b32f-bce4206dd5b1 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:40.641582 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400b928a-6018-4d39-9278-47fc00fad974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:40.642444 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400b928a-6018-4d39-9278-47fc00fad974 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:40.642731 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400b928a-6018-4d39-9278-47fc00fad974 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:40.642998 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400b928a-6018-4d39-9278-47fc00fad974 admin admin] PATH_INFO: `/v3/projects/47df41203da54e04a5a646125282a9ee/users/dba6ea49afe042658b8741317869b128/roles/60af5088793e4379b88821a86552a344` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:40.650906 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400b928a-6018-4d39-9278-47fc00fad974 admin admin] RBAC: Authorizing `identity:create_grant(project_id=47df41203da54e04a5a646125282a9ee, user_id=dba6ea49afe042658b8741317869b128, role_id=60af5088793e4379b88821a86552a344)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:40.653965 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400b928a-6018-4d39-9278-47fc00fad974 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:40.693685 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 816/1639] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:29:40 2022] PUT /identity/v3/projects/47df41203da54e04a5a646125282a9ee/users/dba6ea49afe042658b8741317869b128/roles/60af5088793e4379b88821a86552a344 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:29:40.695213 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400b928a-6018-4d39-9278-47fc00fad974 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:40.797092 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1219c1d8-dd54-4629-9d7d-8549737dd8db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:40.799132 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1219c1d8-dd54-4629-9d7d-8549737dd8db service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:40.829851 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400b928a-6018-4d39-9278-47fc00fad974 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:40.833673 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 824/1640] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:29:40 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:29:40.835143 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2476da-f637-4030-ab1b-da14035639f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:40.835854 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2476da-f637-4030-ab1b-da14035639f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:40.836177 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2476da-f637-4030-ab1b-da14035639f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:40.844208 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2476da-f637-4030-ab1b-da14035639f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:40.844208 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2476da-f637-4030-ab1b-da14035639f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:40.844701 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51212831-df0a-4f4d-9c55-dec32159cb16 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:40.844701 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51212831-df0a-4f4d-9c55-dec32159cb16 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:40.844888 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2476da-f637-4030-ab1b-da14035639f2 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:40.844980 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51212831-df0a-4f4d-9c55-dec32159cb16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:40.854600 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 817/1641] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:29:40 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:29:40.863507 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0f557e-3b16-4882-8c29-1e94a6329f82 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:40.863798 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0f557e-3b16-4882-8c29-1e94a6329f82 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:40.864073 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0f557e-3b16-4882-8c29-1e94a6329f82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:40.872197 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-51212831-df0a-4f4d-9c55-dec32159cb16 None None] MFA Rules not processed for user `fc9129cb9f874b32b99c78349dd58477`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:29:40.873538 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-51212831-df0a-4f4d-9c55-dec32159cb16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:29:40.904647 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-8a0f557e-3b16-4882-8c29-1e94a6329f82 None None] MFA Rules not processed for user `dba6ea49afe042658b8741317869b128`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:29:40.907127 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-8a0f557e-3b16-4882-8c29-1e94a6329f82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:29:40.940218 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 825/1642] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:29:40 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:29:40.946424 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51212831-df0a-4f4d-9c55-dec32159cb16 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:40.959880 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51212831-df0a-4f4d-9c55-dec32159cb16 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:40.966097 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bcde6e-0d60-45af-be15-e2b3e5d7333c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:40.967245 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bcde6e-0d60-45af-be15-e2b3e5d7333c admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:40.968341 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bcde6e-0d60-45af-be15-e2b3e5d7333c admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:40.968966 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bcde6e-0d60-45af-be15-e2b3e5d7333c admin admin] PATH_INFO: `/v3/projects/a5a5bfa5186f41698644470d9ecc8943/users/a63a1d17dc364eed8bb25ed64745b04c/roles/60af5088793e4379b88821a86552a344` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:40.971955 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bcde6e-0d60-45af-be15-e2b3e5d7333c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5a5bfa5186f41698644470d9ecc8943, user_id=a63a1d17dc364eed8bb25ed64745b04c, role_id=60af5088793e4379b88821a86552a344)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:40.974944 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bcde6e-0d60-45af-be15-e2b3e5d7333c admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:40.989213 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 826/1643] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:29:40 2022] PUT /identity/v3/projects/a5a5bfa5186f41698644470d9ecc8943/users/a63a1d17dc364eed8bb25ed64745b04c/roles/60af5088793e4379b88821a86552a344 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:29:40.990587 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bcde6e-0d60-45af-be15-e2b3e5d7333c admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:41.024512 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 818/1644] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:29:40 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:29:41.025994 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dc222b-8c0a-4cc0-bccb-27f71b86fece None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:41.026288 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dc222b-8c0a-4cc0-bccb-27f71b86fece None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:41.026576 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dc222b-8c0a-4cc0-bccb-27f71b86fece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:41.084469 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-a7dc222b-8c0a-4cc0-bccb-27f71b86fece None None] MFA Rules not processed for user `a63a1d17dc364eed8bb25ed64745b04c`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:29:41.086788 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-a7dc222b-8c0a-4cc0-bccb-27f71b86fece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:29:41.106711 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bcde6e-0d60-45af-be15-e2b3e5d7333c admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:41.121778 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820abb28-89d8-4b7c-b472-e88ce0499a8f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c1f878b0dc4d63a4f5dcb12fac1dc1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:41.124740 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820abb28-89d8-4b7c-b472-e88ce0499a8f service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:41.125037 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820abb28-89d8-4b7c-b472-e88ce0499a8f service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:41.125313 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820abb28-89d8-4b7c-b472-e88ce0499a8f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:41.200635 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820abb28-89d8-4b7c-b472-e88ce0499a8f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:41.200893 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 819/1645] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:29:41 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:29:41.201010 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820abb28-89d8-4b7c-b472-e88ce0499a8f service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:41.206589 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dc222b-8c0a-4cc0-bccb-27f71b86fece None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:41.212211 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 827/1646] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:29:40 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:29:41.213631 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9dc3c2-e176-4e62-bbbb-e8d70d5ab79d None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:41.213989 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9dc3c2-e176-4e62-bbbb-e8d70d5ab79d None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:41.214334 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9dc3c2-e176-4e62-bbbb-e8d70d5ab79d None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:41.215852 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 828/1647] 10.222.0.58 () {54 vars in 961 bytes} [Tue May 17 19:29:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:29:41.222513 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3ebf5f-df9e-42a1-9d22-83dd71deaf03 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:41.222799 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3ebf5f-df9e-42a1-9d22-83dd71deaf03 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:41.223070 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3ebf5f-df9e-42a1-9d22-83dd71deaf03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:41.243410 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-db3ebf5f-df9e-42a1-9d22-83dd71deaf03 None None] MFA Rules not processed for user `65c1f878b0dc4d63a4f5dcb12fac1dc1`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:29:41.244727 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-db3ebf5f-df9e-42a1-9d22-83dd71deaf03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:29:41.260344 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 829/1648] 10.222.0.58 () {58 vars in 1058 bytes} [Tue May 17 19:29:41 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 17 19:29:41.262248 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86afe27-c322-4eb6-8d4f-3d6f720da2b3 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:41.262248 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86afe27-c322-4eb6-8d4f-3d6f720da2b3 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:41.262462 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86afe27-c322-4eb6-8d4f-3d6f720da2b3 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:41.268616 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 830/1649] 10.222.0.58 () {54 vars in 912 bytes} [Tue May 17 19:29:41 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:29:41.269979 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae50006-b31c-4f98-8575-22f57054bd12 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:41.270261 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae50006-b31c-4f98-8575-22f57054bd12 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:41.270580 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae50006-b31c-4f98-8575-22f57054bd12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:41.281879 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dc222b-8c0a-4cc0-bccb-27f71b86fece None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:41.286422 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a590f45c-1dc3-4b46-a4dd-d8a318c2503f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd05409b38ae346458abfe41ebda1c59e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:41.287528 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a590f45c-1dc3-4b46-a4dd-d8a318c2503f service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:41.287528 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a590f45c-1dc3-4b46-a4dd-d8a318c2503f service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:41.287720 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a590f45c-1dc3-4b46-a4dd-d8a318c2503f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:41.324946 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-0ae50006-b31c-4f98-8575-22f57054bd12 None None] MFA Rules not processed for user `65c1f878b0dc4d63a4f5dcb12fac1dc1`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:29:41.330722 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-0ae50006-b31c-4f98-8575-22f57054bd12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:29:41.363446 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 831/1650] 10.222.0.58 () {58 vars in 1009 bytes} [Tue May 17 19:29:41 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:29:41.364818 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae50006-b31c-4f98-8575-22f57054bd12 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:41.387550 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae50006-b31c-4f98-8575-22f57054bd12 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:41.393694 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a590f45c-1dc3-4b46-a4dd-d8a318c2503f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:41.395339 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a590f45c-1dc3-4b46-a4dd-d8a318c2503f service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:41.395991 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840096b1-faba-4e44-bc0a-dc6aa397ab44 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd05409b38ae346458abfe41ebda1c59e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:41.396786 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840096b1-faba-4e44-bc0a-dc6aa397ab44 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:41.397067 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840096b1-faba-4e44-bc0a-dc6aa397ab44 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:41.397336 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840096b1-faba-4e44-bc0a-dc6aa397ab44 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:41.411025 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840096b1-faba-4e44-bc0a-dc6aa397ab44 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:41.412944 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840096b1-faba-4e44-bc0a-dc6aa397ab44 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:41.416907 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 820/1651] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:29:41 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:29:41.461712 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 832/1652] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:29:41 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:29:41.612114 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a590f45c-1dc3-4b46-a4dd-d8a318c2503f service cinder] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:41.618192 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840096b1-faba-4e44-bc0a-dc6aa397ab44 service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:41.621463 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a590f45c-1dc3-4b46-a4dd-d8a318c2503f service cinder] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:41.631867 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840096b1-faba-4e44-bc0a-dc6aa397ab44 service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:41.634410 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9363cb26-b117-4907-a003-b47fc02a891b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd05409b38ae346458abfe41ebda1c59e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:41.635513 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9363cb26-b117-4907-a003-b47fc02a891b service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:41.635970 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9363cb26-b117-4907-a003-b47fc02a891b service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:41.636454 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9363cb26-b117-4907-a003-b47fc02a891b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:41.636765 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6790b2eb-a99b-4475-96fb-3937a6a6785c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd05409b38ae346458abfe41ebda1c59e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:41.637479 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6790b2eb-a99b-4475-96fb-3937a6a6785c service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:41.637757 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6790b2eb-a99b-4475-96fb-3937a6a6785c service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:41.638042 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6790b2eb-a99b-4475-96fb-3937a6a6785c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:41.658475 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6790b2eb-a99b-4475-96fb-3937a6a6785c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:41.660980 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6790b2eb-a99b-4475-96fb-3937a6a6785c service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:41.688219 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 833/1653] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:29:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 17 19:29:41.708529 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c20cbc3-c4b1-4df6-b2bf-0b559dcb5c18 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:41.708739 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c20cbc3-c4b1-4df6-b2bf-0b559dcb5c18 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:41.709018 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c20cbc3-c4b1-4df6-b2bf-0b559dcb5c18 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:41.710637 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 834/1654] 10.222.0.58 () {54 vars in 912 bytes} [Tue May 17 19:29:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:29:41.723373 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d1a1bc-88e5-4a95-bc0e-35b8185bac98 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:41.723373 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d1a1bc-88e5-4a95-bc0e-35b8185bac98 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:41.723373 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d1a1bc-88e5-4a95-bc0e-35b8185bac98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:41.739897 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9363cb26-b117-4907-a003-b47fc02a891b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:41.741410 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9363cb26-b117-4907-a003-b47fc02a891b service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:41.766069 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 821/1655] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:29:41 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:29:41.775393 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-e6d1a1bc-88e5-4a95-bc0e-35b8185bac98 None None] MFA Rules not processed for user `1ac193d15f48463c98ff1d97770ed794`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:29:41.781938 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-e6d1a1bc-88e5-4a95-bc0e-35b8185bac98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:29:41.885597 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 835/1656] 10.222.0.58 () {58 vars in 1009 bytes} [Tue May 17 19:29:41 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 17 19:29:41.905449 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9363cb26-b117-4907-a003-b47fc02a891b service cinder] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:41.917071 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9363cb26-b117-4907-a003-b47fc02a891b service cinder] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:41.922605 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695f2c87-5cd3-4dbf-aa5c-b382cee6031b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:41.923667 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695f2c87-5cd3-4dbf-aa5c-b382cee6031b service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:41.924112 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695f2c87-5cd3-4dbf-aa5c-b382cee6031b service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:41.924550 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695f2c87-5cd3-4dbf-aa5c-b382cee6031b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:41.942003 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcebcc0-cf30-4859-ad34-1504d0d19e32 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:41.942003 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcebcc0-cf30-4859-ad34-1504d0d19e32 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:41.942003 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcebcc0-cf30-4859-ad34-1504d0d19e32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:41.952822 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695f2c87-5cd3-4dbf-aa5c-b382cee6031b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:41.954680 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695f2c87-5cd3-4dbf-aa5c-b382cee6031b service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:41.968891 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 822/1657] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:29:41 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:29:41.973058 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695f2c87-5cd3-4dbf-aa5c-b382cee6031b service neutron] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:41.980954 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695f2c87-5cd3-4dbf-aa5c-b382cee6031b service neutron] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:41.985012 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a9721b-d520-482d-8555-7378c34cf985 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd05409b38ae346458abfe41ebda1c59e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:41.985822 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a9721b-d520-482d-8555-7378c34cf985 service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:41.986160 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a9721b-d520-482d-8555-7378c34cf985 service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:41.986497 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a9721b-d520-482d-8555-7378c34cf985 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:42.011305 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-cdcebcc0-cf30-4859-ad34-1504d0d19e32 None None] MFA Rules not processed for user `b652b831bbd24934a21d7aa4c8dfeee7`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:29:42.012557 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-cdcebcc0-cf30-4859-ad34-1504d0d19e32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:29:42.112633 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a9721b-d520-482d-8555-7378c34cf985 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:42.114149 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a9721b-d520-482d-8555-7378c34cf985 service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:42.123946 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 836/1658] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:29:41 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:29:42.128588 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcebcc0-cf30-4859-ad34-1504d0d19e32 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:42.132189 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 823/1659] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:29:41 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:29:42.219602 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcebcc0-cf30-4859-ad34-1504d0d19e32 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:42.225839 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51568dab-120c-40b0-a6c2-fda2d663a4c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:42.226683 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51568dab-120c-40b0-a6c2-fda2d663a4c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:42.226950 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51568dab-120c-40b0-a6c2-fda2d663a4c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:42.227587 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51568dab-120c-40b0-a6c2-fda2d663a4c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:42.228301 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51568dab-120c-40b0-a6c2-fda2d663a4c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:42.230488 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51568dab-120c-40b0-a6c2-fda2d663a4c5 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:42.368908 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 837/1660] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 17 19:29:42 2022] POST /identity/v3/projects => generated 405 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:29:42.370902 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a9721b-d520-482d-8555-7378c34cf985 service cinder] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:42.381948 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a9721b-d520-482d-8555-7378c34cf985 service cinder] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:42.452399 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a90a4c1-95ca-4695-b96f-e88b2b72fff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:42.453222 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a90a4c1-95ca-4695-b96f-e88b2b72fff6 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:42.462263 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a90a4c1-95ca-4695-b96f-e88b2b72fff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:42.462263 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a90a4c1-95ca-4695-b96f-e88b2b72fff6 admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:42.462263 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a90a4c1-95ca-4695-b96f-e88b2b72fff6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:42.462263 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a90a4c1-95ca-4695-b96f-e88b2b72fff6 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:42.574813 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 824/1661] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 17 19:29:42 2022] POST /identity/v3/users => generated 309 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:29:42.576622 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51568dab-120c-40b0-a6c2-fda2d663a4c5 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:42.590664 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51568dab-120c-40b0-a6c2-fda2d663a4c5 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:42.600475 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c791d1fd-a0ce-4c1f-b302-023e50cf243b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:42.602369 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c791d1fd-a0ce-4c1f-b302-023e50cf243b admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:42.602702 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c791d1fd-a0ce-4c1f-b302-023e50cf243b admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:42.602992 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c791d1fd-a0ce-4c1f-b302-023e50cf243b admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:42.603867 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c791d1fd-a0ce-4c1f-b302-023e50cf243b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:42.605987 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c791d1fd-a0ce-4c1f-b302-023e50cf243b admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:42.647303 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 838/1662] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:29:42 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:29:42.647504 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a90a4c1-95ca-4695-b96f-e88b2b72fff6 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:42.659392 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a90a4c1-95ca-4695-b96f-e88b2b72fff6 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:42.674392 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a01e1b9-4f8c-4d75-ad52-df76912c844c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:42.675593 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a01e1b9-4f8c-4d75-ad52-df76912c844c admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:42.675974 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a01e1b9-4f8c-4d75-ad52-df76912c844c admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:42.676352 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a01e1b9-4f8c-4d75-ad52-df76912c844c admin admin] PATH_INFO: `/v3/projects/ea29931dd9f442c2aa79529d6cb18e75/users/1c873f2155834cdfb3870e232ee84697/roles/60af5088793e4379b88821a86552a344` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:42.706707 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a01e1b9-4f8c-4d75-ad52-df76912c844c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea29931dd9f442c2aa79529d6cb18e75, user_id=1c873f2155834cdfb3870e232ee84697, role_id=60af5088793e4379b88821a86552a344)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:42.710126 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a01e1b9-4f8c-4d75-ad52-df76912c844c admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:42.863609 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 825/1663] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:29:42 2022] PUT /identity/v3/projects/ea29931dd9f442c2aa79529d6cb18e75/users/1c873f2155834cdfb3870e232ee84697/roles/60af5088793e4379b88821a86552a344 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 17 19:29:42.867477 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c791d1fd-a0ce-4c1f-b302-023e50cf243b admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:42.882656 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c791d1fd-a0ce-4c1f-b302-023e50cf243b admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:43.033837 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc4bc1a-a1d1-4b41-a573-49de7ddad54f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:43.033837 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc4bc1a-a1d1-4b41-a573-49de7ddad54f admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:43.033837 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc4bc1a-a1d1-4b41-a573-49de7ddad54f admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:43.033837 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc4bc1a-a1d1-4b41-a573-49de7ddad54f admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:43.034839 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc4bc1a-a1d1-4b41-a573-49de7ddad54f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:43.035336 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc4bc1a-a1d1-4b41-a573-49de7ddad54f admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:43.064601 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 839/1664] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:29:42 2022] GET /identity/v3/roles => generated 1420 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:29:43.066633 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a01e1b9-4f8c-4d75-ad52-df76912c844c admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:43.079874 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a01e1b9-4f8c-4d75-ad52-df76912c844c admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:43.088313 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb16e213-16c4-46d9-98e1-af585e1159cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:43.089438 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb16e213-16c4-46d9-98e1-af585e1159cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:43.089881 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb16e213-16c4-46d9-98e1-af585e1159cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:43.090313 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb16e213-16c4-46d9-98e1-af585e1159cc admin admin] PATH_INFO: `/v3/projects/ea29931dd9f442c2aa79529d6cb18e75/users/1c873f2155834cdfb3870e232ee84697/roles/60af5088793e4379b88821a86552a344` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:43.092651 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb16e213-16c4-46d9-98e1-af585e1159cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea29931dd9f442c2aa79529d6cb18e75, user_id=1c873f2155834cdfb3870e232ee84697, role_id=60af5088793e4379b88821a86552a344)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:43.095825 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb16e213-16c4-46d9-98e1-af585e1159cc admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:43.122902 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 826/1665] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:29:43 2022] PUT /identity/v3/projects/ea29931dd9f442c2aa79529d6cb18e75/users/1c873f2155834cdfb3870e232ee84697/roles/60af5088793e4379b88821a86552a344 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:29:43.153026 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d8de48-915b-473f-806a-a362ab48d16e None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:43.153026 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d8de48-915b-473f-806a-a362ab48d16e None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:43.153026 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d8de48-915b-473f-806a-a362ab48d16e None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:43.153026 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 840/1666] 10.222.0.58 () {54 vars in 960 bytes} [Tue May 17 19:29:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:29:43.157564 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a78c5f-83a8-4912-8170-de71c172fafd None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:43.157868 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a78c5f-83a8-4912-8170-de71c172fafd None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:43.158150 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a78c5f-83a8-4912-8170-de71c172fafd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:43.193759 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-a5a78c5f-83a8-4912-8170-de71c172fafd None None] MFA Rules not processed for user `06538aab5f194af7b7c677d6b94aef2a`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:29:43.195112 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-a5a78c5f-83a8-4912-8170-de71c172fafd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:29:43.518270 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 827/1667] 10.222.0.58 () {58 vars in 1057 bytes} [Tue May 17 19:29:43 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 17 19:29:43.559776 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d8de48-915b-473f-806a-a362ab48d16e None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:43.559776 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d8de48-915b-473f-806a-a362ab48d16e None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:43.559776 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9976d667-5614-434e-bbb2-05ec2859283a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06538aab5f194af7b7c677d6b94aef2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:43.559776 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9976d667-5614-434e-bbb2-05ec2859283a service placement] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:43.567617 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9976d667-5614-434e-bbb2-05ec2859283a service placement] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:43.567617 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9976d667-5614-434e-bbb2-05ec2859283a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:43.567617 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-9976d667-5614-434e-bbb2-05ec2859283a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:29:43.567617 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9976d667-5614-434e-bbb2-05ec2859283a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:43.567617 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9976d667-5614-434e-bbb2-05ec2859283a service placement] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:43.567617 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 841/1668] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:29:43 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 17 19:29:44.169784 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a78c5f-83a8-4912-8170-de71c172fafd None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:44.398687 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a78c5f-83a8-4912-8170-de71c172fafd None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:44.422716 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defef58e-2010-43eb-bcb0-96100236cec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:44.423722 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defef58e-2010-43eb-bcb0-96100236cec3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:44.430332 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defef58e-2010-43eb-bcb0-96100236cec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:44.430332 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defef58e-2010-43eb-bcb0-96100236cec3 admin admin] PATH_INFO: `/v3/users/6f6b0266e7914eaf9f1103addc78c485` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:44.430332 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defef58e-2010-43eb-bcb0-96100236cec3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f6b0266e7914eaf9f1103addc78c485)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:44.431062 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defef58e-2010-43eb-bcb0-96100236cec3 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:44.777853 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-defef58e-2010-43eb-bcb0-96100236cec3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f6b0266e7914eaf9f1103addc78c485'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:29:44.792475 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-defef58e-2010-43eb-bcb0-96100236cec3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f6b0266e7914eaf9f1103addc78c485'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:29:44.834640 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-defef58e-2010-43eb-bcb0-96100236cec3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f6b0266e7914eaf9f1103addc78c485'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:29:44.834640 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-defef58e-2010-43eb-bcb0-96100236cec3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f6b0266e7914eaf9f1103addc78c485'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:29:44.834640 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-defef58e-2010-43eb-bcb0-96100236cec3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f6b0266e7914eaf9f1103addc78c485'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:29:44.943885 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 828/1669] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 17 19:29:44 2022] DELETE /identity/v3/users/6f6b0266e7914eaf9f1103addc78c485 => generated 0 bytes in 777 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:29:44.947851 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416b422d-faf0-43eb-a3bf-71eab99d9d1d None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:44.948362 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416b422d-faf0-43eb-a3bf-71eab99d9d1d None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:44.948439 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416b422d-faf0-43eb-a3bf-71eab99d9d1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:44.984876 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defef58e-2010-43eb-bcb0-96100236cec3 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:44.986381 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-defef58e-2010-43eb-bcb0-96100236cec3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:29:45.020384 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-416b422d-faf0-43eb-a3bf-71eab99d9d1d None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:29:45.020384 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-416b422d-faf0-43eb-a3bf-71eab99d9d1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:29:45.251890 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 842/1670] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:29:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:29:45.276833 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defef58e-2010-43eb-bcb0-96100236cec3 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:45.280765 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45660713-1645-46c6-be76-9a8bdf495c21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:45.282142 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45660713-1645-46c6-be76-9a8bdf495c21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:45.282595 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45660713-1645-46c6-be76-9a8bdf495c21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:45.283166 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45660713-1645-46c6-be76-9a8bdf495c21 admin admin] PATH_INFO: `/v3/users/61876ef673e14d8f849099c9972c9e2f` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:45.284854 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45660713-1645-46c6-be76-9a8bdf495c21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=61876ef673e14d8f849099c9972c9e2f)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:45.287120 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45660713-1645-46c6-be76-9a8bdf495c21 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:45.289478 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416b422d-faf0-43eb-a3bf-71eab99d9d1d None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:45.290920 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-416b422d-faf0-43eb-a3bf-71eab99d9d1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:29:45.385592 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416b422d-faf0-43eb-a3bf-71eab99d9d1d None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:45.391458 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44df774b-73de-4b89-a754-dc59f9cfd720 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:45.392566 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44df774b-73de-4b89-a754-dc59f9cfd720 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:45.393172 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44df774b-73de-4b89-a754-dc59f9cfd720 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:45.393727 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44df774b-73de-4b89-a754-dc59f9cfd720 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:45.406191 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44df774b-73de-4b89-a754-dc59f9cfd720 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:45.409100 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44df774b-73de-4b89-a754-dc59f9cfd720 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:45.423030 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 843/1671] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:29:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 17 19:29:45.565671 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-45660713-1645-46c6-be76-9a8bdf495c21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61876ef673e14d8f849099c9972c9e2f'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:29:45.595633 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-45660713-1645-46c6-be76-9a8bdf495c21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61876ef673e14d8f849099c9972c9e2f'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:29:45.618082 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-45660713-1645-46c6-be76-9a8bdf495c21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61876ef673e14d8f849099c9972c9e2f'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:29:45.618862 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-45660713-1645-46c6-be76-9a8bdf495c21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61876ef673e14d8f849099c9972c9e2f'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:29:45.619526 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-45660713-1645-46c6-be76-9a8bdf495c21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61876ef673e14d8f849099c9972c9e2f'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:29:45.718691 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023088f6-e3dd-4365-b273-d01f3de93968 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:45.719163 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023088f6-e3dd-4365-b273-d01f3de93968 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:45.719616 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023088f6-e3dd-4365-b273-d01f3de93968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:45.725500 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 829/1672] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 17 19:29:44 2022] DELETE /identity/v3/users/61876ef673e14d8f849099c9972c9e2f => generated 0 bytes in 730 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:29:45.755460 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-023088f6-e3dd-4365-b273-d01f3de93968 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:29:45.759375 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-023088f6-e3dd-4365-b273-d01f3de93968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:29:45.966627 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 844/1673] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:29:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:29:45.972848 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d58220-7b72-46f1-92c9-2b792b08fbdf None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:45.976818 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d58220-7b72-46f1-92c9-2b792b08fbdf None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:45.976818 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d58220-7b72-46f1-92c9-2b792b08fbdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:45.990554 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023088f6-e3dd-4365-b273-d01f3de93968 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:45.992888 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-023088f6-e3dd-4365-b273-d01f3de93968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:29:46.066307 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-d3d58220-7b72-46f1-92c9-2b792b08fbdf None None] MFA Rules not processed for user `1c873f2155834cdfb3870e232ee84697`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:29:46.068761 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-d3d58220-7b72-46f1-92c9-2b792b08fbdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:29:46.109708 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023088f6-e3dd-4365-b273-d01f3de93968 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:46.114839 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05eae4a2-c244-4c88-9609-0bf24ebe64ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:46.115824 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eae4a2-c244-4c88-9609-0bf24ebe64ac service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:46.116272 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eae4a2-c244-4c88-9609-0bf24ebe64ac service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:46.116942 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eae4a2-c244-4c88-9609-0bf24ebe64ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:46.135029 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05eae4a2-c244-4c88-9609-0bf24ebe64ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:46.138510 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05eae4a2-c244-4c88-9609-0bf24ebe64ac service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:46.168948 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 845/1674] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:29:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:29:46.257035 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 830/1675] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:29:45 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:29:46.706529 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05eae4a2-c244-4c88-9609-0bf24ebe64ac service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:46.708429 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-05eae4a2-c244-4c88-9609-0bf24ebe64ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:29:46.742107 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05eae4a2-c244-4c88-9609-0bf24ebe64ac service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:46.778252 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785acc40-3797-45ac-9fae-be0ea34ef15d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:46.783975 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785acc40-3797-45ac-9fae-be0ea34ef15d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:46.783975 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785acc40-3797-45ac-9fae-be0ea34ef15d admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:46.783975 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785acc40-3797-45ac-9fae-be0ea34ef15d admin admin] PATH_INFO: `/v3/projects/947298bea89442fea518ea0a65c1e7cb` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:46.792477 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785acc40-3797-45ac-9fae-be0ea34ef15d admin admin] RBAC: Authorizing `identity:delete_project(project_id=947298bea89442fea518ea0a65c1e7cb)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:46.805691 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785acc40-3797-45ac-9fae-be0ea34ef15d admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:47.035181 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-785acc40-3797-45ac-9fae-be0ea34ef15d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '947298bea89442fea518ea0a65c1e7cb', 'request_id': 'req-785acc40-3797-45ac-9fae-be0ea34ef15d'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:29:47.055929 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-785acc40-3797-45ac-9fae-be0ea34ef15d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '947298bea89442fea518ea0a65c1e7cb', 'request_id': 'req-785acc40-3797-45ac-9fae-be0ea34ef15d'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:29:47.166938 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-785acc40-3797-45ac-9fae-be0ea34ef15d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '947298bea89442fea518ea0a65c1e7cb', 'request_id': 'req-785acc40-3797-45ac-9fae-be0ea34ef15d'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:29:47.202853 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-785acc40-3797-45ac-9fae-be0ea34ef15d admin admin] The token cache is being invalidate because project 947298bea89442fea518ea0a65c1e7cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 17 19:29:47.203275 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-785acc40-3797-45ac-9fae-be0ea34ef15d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:29:47.204128 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-785acc40-3797-45ac-9fae-be0ea34ef15d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:29:47.211262 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d58220-7b72-46f1-92c9-2b792b08fbdf None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:47.211846 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 846/1676] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 17 19:29:46 2022] DELETE /identity/v3/projects/947298bea89442fea518ea0a65c1e7cb => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:29:47.225495 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d58220-7b72-46f1-92c9-2b792b08fbdf None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:47.301853 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e956a46f-2e06-46a4-851d-3eeed28046bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:47.302960 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e956a46f-2e06-46a4-851d-3eeed28046bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:47.303243 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e956a46f-2e06-46a4-851d-3eeed28046bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:47.303556 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e956a46f-2e06-46a4-851d-3eeed28046bd admin admin] PATH_INFO: `/v3/users/dba6ea49afe042658b8741317869b128` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:47.304879 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e956a46f-2e06-46a4-851d-3eeed28046bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=dba6ea49afe042658b8741317869b128)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:47.306782 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e956a46f-2e06-46a4-851d-3eeed28046bd admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:47.402445 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785acc40-3797-45ac-9fae-be0ea34ef15d admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:47.403920 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-785acc40-3797-45ac-9fae-be0ea34ef15d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:29:47.431651 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785acc40-3797-45ac-9fae-be0ea34ef15d admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:47.438992 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93bda68-f0a6-4d52-a14a-de1303981bf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:47.440588 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93bda68-f0a6-4d52-a14a-de1303981bf6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:47.441131 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93bda68-f0a6-4d52-a14a-de1303981bf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:47.441809 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93bda68-f0a6-4d52-a14a-de1303981bf6 admin admin] PATH_INFO: `/v3/projects/7fec6239168e4f15946ccff5cd48d145` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:47.447207 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93bda68-f0a6-4d52-a14a-de1303981bf6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7fec6239168e4f15946ccff5cd48d145)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:47.460673 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93bda68-f0a6-4d52-a14a-de1303981bf6 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:47.554259 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-e956a46f-2e06-46a4-851d-3eeed28046bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dba6ea49afe042658b8741317869b128'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:29:47.570779 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-e956a46f-2e06-46a4-851d-3eeed28046bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dba6ea49afe042658b8741317869b128'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:29:47.590889 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-e956a46f-2e06-46a4-851d-3eeed28046bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dba6ea49afe042658b8741317869b128'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:29:47.591962 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-e956a46f-2e06-46a4-851d-3eeed28046bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dba6ea49afe042658b8741317869b128'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:29:47.592749 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-e956a46f-2e06-46a4-851d-3eeed28046bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dba6ea49afe042658b8741317869b128'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:29:47.643958 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c93bda68-f0a6-4d52-a14a-de1303981bf6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7fec6239168e4f15946ccff5cd48d145', 'request_id': 'req-c93bda68-f0a6-4d52-a14a-de1303981bf6'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:29:47.660281 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 831/1677] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 17 19:29:47 2022] DELETE /identity/v3/users/dba6ea49afe042658b8741317869b128 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:29:47.665367 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c93bda68-f0a6-4d52-a14a-de1303981bf6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7fec6239168e4f15946ccff5cd48d145', 'request_id': 'req-c93bda68-f0a6-4d52-a14a-de1303981bf6'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:29:47.739010 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c93bda68-f0a6-4d52-a14a-de1303981bf6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7fec6239168e4f15946ccff5cd48d145', 'request_id': 'req-c93bda68-f0a6-4d52-a14a-de1303981bf6'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:29:47.762538 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c93bda68-f0a6-4d52-a14a-de1303981bf6 admin admin] The token cache is being invalidate because project 7fec6239168e4f15946ccff5cd48d145 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 17 19:29:47.765739 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c93bda68-f0a6-4d52-a14a-de1303981bf6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:29:47.773252 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c93bda68-f0a6-4d52-a14a-de1303981bf6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:29:47.779555 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 847/1678] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 17 19:29:47 2022] DELETE /identity/v3/projects/7fec6239168e4f15946ccff5cd48d145 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:29:47.783805 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e8d578-e0ed-4657-af69-6685a1151878 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:47.793613 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e8d578-e0ed-4657-af69-6685a1151878 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:47.793613 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e8d578-e0ed-4657-af69-6685a1151878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:47.852179 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-c2e8d578-e0ed-4657-af69-6685a1151878 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:29:47.855531 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-c2e8d578-e0ed-4657-af69-6685a1151878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:29:48.011156 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5a5606-0705-4a39-8b0f-e18a5a96d300 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:48.011701 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5a5606-0705-4a39-8b0f-e18a5a96d300 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:48.012224 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5a5606-0705-4a39-8b0f-e18a5a96d300 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:48.014979 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 832/1679] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:29:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:29:48.047135 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-6b5a5606-0705-4a39-8b0f-e18a5a96d300 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:29:48.048660 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-6b5a5606-0705-4a39-8b0f-e18a5a96d300 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:29:48.058423 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 848/1680] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:29:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:29:48.062955 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e8d578-e0ed-4657-af69-6685a1151878 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:48.073876 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e8d578-e0ed-4657-af69-6685a1151878 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:48.079764 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0db04f-28f4-4f2e-a161-fba42c4fff5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:48.080746 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0db04f-28f4-4f2e-a161-fba42c4fff5f admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:48.081806 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0db04f-28f4-4f2e-a161-fba42c4fff5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:48.082235 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0db04f-28f4-4f2e-a161-fba42c4fff5f admin admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:48.083183 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0db04f-28f4-4f2e-a161-fba42c4fff5f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:29:48.083521 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0db04f-28f4-4f2e-a161-fba42c4fff5f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:48.085397 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0db04f-28f4-4f2e-a161-fba42c4fff5f admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:48.115560 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 833/1681] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 17 19:29:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:29:48.116771 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5a5606-0705-4a39-8b0f-e18a5a96d300 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:48.131887 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5a5606-0705-4a39-8b0f-e18a5a96d300 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:48.136934 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6332fef-0590-46c4-aba2-81926a60551f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:48.137832 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6332fef-0590-46c4-aba2-81926a60551f admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:48.138253 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6332fef-0590-46c4-aba2-81926a60551f admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:48.141003 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6332fef-0590-46c4-aba2-81926a60551f admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:48.141870 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6332fef-0590-46c4-aba2-81926a60551f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:48.144300 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6332fef-0590-46c4-aba2-81926a60551f admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:48.271976 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0db04f-28f4-4f2e-a161-fba42c4fff5f admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:48.283244 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 849/1682] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 17 19:29:48 2022] POST /identity/v3/projects => generated 367 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:29:48.293970 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0db04f-28f4-4f2e-a161-fba42c4fff5f admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:48.427697 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ec0deb-9ea5-4d31-b081-b03092a70e1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:48.429411 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ec0deb-9ea5-4d31-b081-b03092a70e1a admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:48.430130 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ec0deb-9ea5-4d31-b081-b03092a70e1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:48.430474 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ec0deb-9ea5-4d31-b081-b03092a70e1a admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:48.431199 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ec0deb-9ea5-4d31-b081-b03092a70e1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:48.434535 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ec0deb-9ea5-4d31-b081-b03092a70e1a admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:48.529600 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 834/1683] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 17 19:29:48 2022] POST /identity/v3/users => generated 284 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:29:48.535051 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6332fef-0590-46c4-aba2-81926a60551f admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:48.565070 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6332fef-0590-46c4-aba2-81926a60551f admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:48.590804 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cbd009-fa45-4240-a18e-f81b66b86dfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:48.591896 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cbd009-fa45-4240-a18e-f81b66b86dfd admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:48.592357 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cbd009-fa45-4240-a18e-f81b66b86dfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:48.593419 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cbd009-fa45-4240-a18e-f81b66b86dfd admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:48.595178 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cbd009-fa45-4240-a18e-f81b66b86dfd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:48.597765 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cbd009-fa45-4240-a18e-f81b66b86dfd admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:48.628970 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 850/1684] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:29:48 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:29:48.632939 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ec0deb-9ea5-4d31-b081-b03092a70e1a admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:48.647127 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ec0deb-9ea5-4d31-b081-b03092a70e1a admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:48.659656 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a1a642-66b2-4805-8f80-a388004f7136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:48.681130 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a1a642-66b2-4805-8f80-a388004f7136 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:48.681130 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a1a642-66b2-4805-8f80-a388004f7136 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:48.681130 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a1a642-66b2-4805-8f80-a388004f7136 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:48.681130 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a1a642-66b2-4805-8f80-a388004f7136 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:48.681130 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a1a642-66b2-4805-8f80-a388004f7136 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:48.683532 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 835/1685] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:29:48 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:29:48.684669 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cbd009-fa45-4240-a18e-f81b66b86dfd admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:48.698901 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cbd009-fa45-4240-a18e-f81b66b86dfd admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:48.705601 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5b20ab-7911-4ff5-9753-5853e993d766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:48.710516 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5b20ab-7911-4ff5-9753-5853e993d766 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:48.710516 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5b20ab-7911-4ff5-9753-5853e993d766 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:48.710516 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5b20ab-7911-4ff5-9753-5853e993d766 admin admin] PATH_INFO: `/v3/projects/95b5d609c5964ab6a459ce08fea64ce4/users/8b8bfa2f51cb4333897752cce6b5026b/roles/114e6a16e63447d1924707ace5aace25` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:48.785948 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5b20ab-7911-4ff5-9753-5853e993d766 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95b5d609c5964ab6a459ce08fea64ce4, user_id=8b8bfa2f51cb4333897752cce6b5026b, role_id=114e6a16e63447d1924707ace5aace25)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:48.791374 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5b20ab-7911-4ff5-9753-5853e993d766 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:48.793492 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a1a642-66b2-4805-8f80-a388004f7136 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:48.796570 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-53a1a642-66b2-4805-8f80-a388004f7136 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:29:48.823275 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a1a642-66b2-4805-8f80-a388004f7136 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:48.833554 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d15d9c1-c0c1-4f34-aed6-c698456fe367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:48.840467 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d15d9c1-c0c1-4f34-aed6-c698456fe367 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:48.841108 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d15d9c1-c0c1-4f34-aed6-c698456fe367 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:48.842065 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d15d9c1-c0c1-4f34-aed6-c698456fe367 admin admin] PATH_INFO: `/v3/users/5ce69eb7e8cf40539ef9dc09d8a22f38` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:48.845021 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d15d9c1-c0c1-4f34-aed6-c698456fe367 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ce69eb7e8cf40539ef9dc09d8a22f38)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:48.852918 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d15d9c1-c0c1-4f34-aed6-c698456fe367 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:48.883441 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 851/1686] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:29:48 2022] PUT /identity/v3/projects/95b5d609c5964ab6a459ce08fea64ce4/users/8b8bfa2f51cb4333897752cce6b5026b/roles/114e6a16e63447d1924707ace5aace25 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 17 19:29:48.914593 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb3d823-8954-4950-98cf-dd47a5e8f9b4 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:48.915656 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb3d823-8954-4950-98cf-dd47a5e8f9b4 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:48.916041 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb3d823-8954-4950-98cf-dd47a5e8f9b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:49.019146 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-7bb3d823-8954-4950-98cf-dd47a5e8f9b4 None None] MFA Rules not processed for user `8b8bfa2f51cb4333897752cce6b5026b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:29:49.021110 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-7bb3d823-8954-4950-98cf-dd47a5e8f9b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:29:49.168178 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 852/1687] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:29:48 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:29:49.177782 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb3d823-8954-4950-98cf-dd47a5e8f9b4 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:49.179400 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-7bb3d823-8954-4950-98cf-dd47a5e8f9b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:29:49.223026 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-6d15d9c1-c0c1-4f34-aed6-c698456fe367 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ce69eb7e8cf40539ef9dc09d8a22f38'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:29:49.240550 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-6d15d9c1-c0c1-4f34-aed6-c698456fe367 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ce69eb7e8cf40539ef9dc09d8a22f38'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:29:49.265284 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-6d15d9c1-c0c1-4f34-aed6-c698456fe367 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ce69eb7e8cf40539ef9dc09d8a22f38'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:29:49.267686 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-6d15d9c1-c0c1-4f34-aed6-c698456fe367 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ce69eb7e8cf40539ef9dc09d8a22f38'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:29:49.268222 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-6d15d9c1-c0c1-4f34-aed6-c698456fe367 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ce69eb7e8cf40539ef9dc09d8a22f38'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:29:49.303992 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb3d823-8954-4950-98cf-dd47a5e8f9b4 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:49.323635 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c4d58e-69c3-4c03-a0b0-917359e27ac0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd05409b38ae346458abfe41ebda1c59e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:49.324742 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c4d58e-69c3-4c03-a0b0-917359e27ac0 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:49.325099 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c4d58e-69c3-4c03-a0b0-917359e27ac0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:49.325377 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c4d58e-69c3-4c03-a0b0-917359e27ac0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:49.327943 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b5c4d58e-69c3-4c03-a0b0-917359e27ac0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:29:49.356873 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 836/1688] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 17 19:29:48 2022] DELETE /identity/v3/users/5ce69eb7e8cf40539ef9dc09d8a22f38 => generated 0 bytes in 564 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:29:49.358929 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d15d9c1-c0c1-4f34-aed6-c698456fe367 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:49.362612 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-6d15d9c1-c0c1-4f34-aed6-c698456fe367 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:29:49.365135 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c4d58e-69c3-4c03-a0b0-917359e27ac0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:49.366818 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c4d58e-69c3-4c03-a0b0-917359e27ac0 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:49.390207 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 853/1689] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:29:49 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:29:49.396309 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5c2b10-68ce-43f2-a71f-3cc8679072a3 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:49.396641 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5c2b10-68ce-43f2-a71f-3cc8679072a3 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:49.396910 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5c2b10-68ce-43f2-a71f-3cc8679072a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:49.445596 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-cb5c2b10-68ce-43f2-a71f-3cc8679072a3 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:29:49.450095 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-cb5c2b10-68ce-43f2-a71f-3cc8679072a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:29:49.535325 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d15d9c1-c0c1-4f34-aed6-c698456fe367 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:49.549447 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e8f64f-67f4-4073-bb9b-e7e56f75349e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:49.550729 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e8f64f-67f4-4073-bb9b-e7e56f75349e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:49.551143 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e8f64f-67f4-4073-bb9b-e7e56f75349e admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:49.551601 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e8f64f-67f4-4073-bb9b-e7e56f75349e admin admin] PATH_INFO: `/v3/projects/47df41203da54e04a5a646125282a9ee` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:49.555812 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e8f64f-67f4-4073-bb9b-e7e56f75349e admin admin] RBAC: Authorizing `identity:delete_project(project_id=47df41203da54e04a5a646125282a9ee)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:49.557027 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: WARNING py.warnings [None req-10e8f64f-67f4-4073-bb9b-e7e56f75349e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 17 19:29:49.557027 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: warnings.warn(msg) May 17 19:29:49.557027 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: May 17 19:29:49.567046 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e8f64f-67f4-4073-bb9b-e7e56f75349e admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:49.624447 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 854/1690] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:29:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:29:49.641544 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5c2b10-68ce-43f2-a71f-3cc8679072a3 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:49.644811 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-cb5c2b10-68ce-43f2-a71f-3cc8679072a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:29:49.740832 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-10e8f64f-67f4-4073-bb9b-e7e56f75349e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47df41203da54e04a5a646125282a9ee', 'request_id': 'req-10e8f64f-67f4-4073-bb9b-e7e56f75349e'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:29:49.776326 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-10e8f64f-67f4-4073-bb9b-e7e56f75349e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47df41203da54e04a5a646125282a9ee', 'request_id': 'req-10e8f64f-67f4-4073-bb9b-e7e56f75349e'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:29:49.790763 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-10e8f64f-67f4-4073-bb9b-e7e56f75349e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47df41203da54e04a5a646125282a9ee', 'request_id': 'req-10e8f64f-67f4-4073-bb9b-e7e56f75349e'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:29:49.804071 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5c2b10-68ce-43f2-a71f-3cc8679072a3 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:49.896991 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-10e8f64f-67f4-4073-bb9b-e7e56f75349e admin admin] The token cache is being invalidate because project 47df41203da54e04a5a646125282a9ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 17 19:29:49.897471 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-10e8f64f-67f4-4073-bb9b-e7e56f75349e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:29:49.899047 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-10e8f64f-67f4-4073-bb9b-e7e56f75349e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:29:49.901425 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 837/1691] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 17 19:29:49 2022] DELETE /identity/v3/projects/47df41203da54e04a5a646125282a9ee => generated 0 bytes in 543 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:29:49.909554 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3af0938-5d77-44b6-bcbf-9f026efbaa6b None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:49.909976 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3af0938-5d77-44b6-bcbf-9f026efbaa6b None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:49.910359 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3af0938-5d77-44b6-bcbf-9f026efbaa6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:49.953214 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-a3af0938-5d77-44b6-bcbf-9f026efbaa6b None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:29:49.954454 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d51532c-92ef-438b-b1fd-1d61d1832f64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:49.960205 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-a3af0938-5d77-44b6-bcbf-9f026efbaa6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:29:49.960346 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d51532c-92ef-438b-b1fd-1d61d1832f64 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:49.960346 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d51532c-92ef-438b-b1fd-1d61d1832f64 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:49.960346 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d51532c-92ef-438b-b1fd-1d61d1832f64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:49.963065 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-3d51532c-92ef-438b-b1fd-1d61d1832f64 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:29:50.095376 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 838/1692] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:29:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:29:50.105704 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8715eb4-4d60-464b-a973-5e86d10be5ed None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:50.107752 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8715eb4-4d60-464b-a973-5e86d10be5ed None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:50.109905 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8715eb4-4d60-464b-a973-5e86d10be5ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:50.179822 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d51532c-92ef-438b-b1fd-1d61d1832f64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:50.188430 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d51532c-92ef-438b-b1fd-1d61d1832f64 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:50.209756 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-f8715eb4-4d60-464b-a973-5e86d10be5ed None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:29:50.211340 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-f8715eb4-4d60-464b-a973-5e86d10be5ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:29:50.240877 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 855/1693] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:29:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 600 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 17 19:29:50.245768 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 839/1694] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:29:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:29:50.250781 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d51532c-92ef-438b-b1fd-1d61d1832f64 service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:50.268735 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d51532c-92ef-438b-b1fd-1d61d1832f64 service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:50.277872 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bfe42c-a4a4-4cba-bcf5-a4a57ff33063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:50.278491 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bfe42c-a4a4-4cba-bcf5-a4a57ff33063 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:50.278770 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bfe42c-a4a4-4cba-bcf5-a4a57ff33063 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:50.279041 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bfe42c-a4a4-4cba-bcf5-a4a57ff33063 admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:50.280441 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bfe42c-a4a4-4cba-bcf5-a4a57ff33063 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:29:50.282164 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bfe42c-a4a4-4cba-bcf5-a4a57ff33063 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:50.282471 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bfe42c-a4a4-4cba-bcf5-a4a57ff33063 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:50.303514 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 856/1695] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 17 19:29:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:29:50.305575 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8715eb4-4d60-464b-a973-5e86d10be5ed None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:50.321469 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8715eb4-4d60-464b-a973-5e86d10be5ed None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:50.335957 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc32326-0bc4-4379-a741-61ee16d30cab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:50.337022 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc32326-0bc4-4379-a741-61ee16d30cab admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:50.337819 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc32326-0bc4-4379-a741-61ee16d30cab admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:50.338103 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc32326-0bc4-4379-a741-61ee16d30cab admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:50.342028 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc32326-0bc4-4379-a741-61ee16d30cab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:50.347644 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc32326-0bc4-4379-a741-61ee16d30cab admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:50.460825 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bfe42c-a4a4-4cba-bcf5-a4a57ff33063 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:50.462946 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-98bfe42c-a4a4-4cba-bcf5-a4a57ff33063 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:29:50.490586 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bfe42c-a4a4-4cba-bcf5-a4a57ff33063 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:50.500181 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6391637-da73-4de4-8a3b-e0b1ba4d3fec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:50.501236 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6391637-da73-4de4-8a3b-e0b1ba4d3fec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:50.501522 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6391637-da73-4de4-8a3b-e0b1ba4d3fec admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:50.501794 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6391637-da73-4de4-8a3b-e0b1ba4d3fec admin admin] PATH_INFO: `/v3/users/b652b831bbd24934a21d7aa4c8dfeee7` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:50.505532 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6391637-da73-4de4-8a3b-e0b1ba4d3fec admin admin] RBAC: Authorizing `identity:delete_user(user_id=b652b831bbd24934a21d7aa4c8dfeee7)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:50.528566 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 840/1696] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 17 19:29:50 2022] POST /identity/v3/projects => generated 385 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:29:50.531313 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: WARNING py.warnings [None req-e6391637-da73-4de4-8a3b-e0b1ba4d3fec admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 17 19:29:50.531313 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: warnings.warn(msg) May 17 19:29:50.531313 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: May 17 19:29:50.536522 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc32326-0bc4-4379-a741-61ee16d30cab admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:50.537915 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6391637-da73-4de4-8a3b-e0b1ba4d3fec admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:50.760573 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc32326-0bc4-4379-a741-61ee16d30cab admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:50.782033 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6734e93-937a-433b-9cfb-2306c466688d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:50.783053 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6734e93-937a-433b-9cfb-2306c466688d admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:50.783476 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6734e93-937a-433b-9cfb-2306c466688d admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:50.783912 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6734e93-937a-433b-9cfb-2306c466688d admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:50.784958 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6734e93-937a-433b-9cfb-2306c466688d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:50.787081 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6734e93-937a-433b-9cfb-2306c466688d admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:50.909401 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-e6391637-da73-4de4-8a3b-e0b1ba4d3fec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b652b831bbd24934a21d7aa4c8dfeee7'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:29:50.910932 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-e6391637-da73-4de4-8a3b-e0b1ba4d3fec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b652b831bbd24934a21d7aa4c8dfeee7'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:29:50.954264 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-e6391637-da73-4de4-8a3b-e0b1ba4d3fec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b652b831bbd24934a21d7aa4c8dfeee7'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:29:50.966994 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 841/1697] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 17 19:29:50 2022] POST /identity/v3/users => generated 293 bytes in 437 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:29:50.972825 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6734e93-937a-433b-9cfb-2306c466688d admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:50.975672 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-c6734e93-937a-433b-9cfb-2306c466688d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:29:50.997286 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6734e93-937a-433b-9cfb-2306c466688d admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:51.003558 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e855f14-0843-4892-9c27-4384ef0c3264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:51.006911 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e855f14-0843-4892-9c27-4384ef0c3264 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:51.006911 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e855f14-0843-4892-9c27-4384ef0c3264 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:51.006911 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e855f14-0843-4892-9c27-4384ef0c3264 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:51.006911 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e855f14-0843-4892-9c27-4384ef0c3264 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:51.007598 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e855f14-0843-4892-9c27-4384ef0c3264 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:51.019806 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 842/1698] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:29:50 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:29:51.023849 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290bfdbf-0872-4537-98e2-796a7a5f52a1 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:51.025804 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290bfdbf-0872-4537-98e2-796a7a5f52a1 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:51.026256 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290bfdbf-0872-4537-98e2-796a7a5f52a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:51.060040 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-290bfdbf-0872-4537-98e2-796a7a5f52a1 None None] MFA Rules not processed for user `b171cf633e6e437fb509857b0a2962e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:29:51.062056 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-290bfdbf-0872-4537-98e2-796a7a5f52a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:29:51.086290 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-e6391637-da73-4de4-8a3b-e0b1ba4d3fec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b652b831bbd24934a21d7aa4c8dfeee7'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:29:51.110639 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-e6391637-da73-4de4-8a3b-e0b1ba4d3fec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b652b831bbd24934a21d7aa4c8dfeee7'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:29:51.119275 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 857/1699] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 17 19:29:50 2022] DELETE /identity/v3/users/b652b831bbd24934a21d7aa4c8dfeee7 => generated 0 bytes in 659 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:29:51.120364 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6391637-da73-4de4-8a3b-e0b1ba4d3fec admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:51.167831 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 843/1700] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:29:51 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:29:51.182385 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9610f57e-288e-4c7c-9e88-2a8f32ec297f None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:51.182675 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9610f57e-288e-4c7c-9e88-2a8f32ec297f None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:51.182956 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9610f57e-288e-4c7c-9e88-2a8f32ec297f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:51.222937 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6391637-da73-4de4-8a3b-e0b1ba4d3fec admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:51.234191 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-9610f57e-288e-4c7c-9e88-2a8f32ec297f None None] MFA Rules not processed for user `b171cf633e6e437fb509857b0a2962e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:29:51.236789 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-9610f57e-288e-4c7c-9e88-2a8f32ec297f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:29:51.250105 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d375c1a7-681c-4450-8aaf-ba074b3af3b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:51.255775 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d375c1a7-681c-4450-8aaf-ba074b3af3b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:51.256202 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d375c1a7-681c-4450-8aaf-ba074b3af3b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:51.259193 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d375c1a7-681c-4450-8aaf-ba074b3af3b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:51.259193 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d375c1a7-681c-4450-8aaf-ba074b3af3b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:51.259698 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d375c1a7-681c-4450-8aaf-ba074b3af3b0 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:51.288236 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 858/1701] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:29:51 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:29:51.295009 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d375c1a7-681c-4450-8aaf-ba074b3af3b0 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:51.310377 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d375c1a7-681c-4450-8aaf-ba074b3af3b0 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:51.316555 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7290bf20-15ce-405b-b1de-64073c670260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:51.322524 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7290bf20-15ce-405b-b1de-64073c670260 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:51.322902 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7290bf20-15ce-405b-b1de-64073c670260 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:51.323445 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7290bf20-15ce-405b-b1de-64073c670260 admin admin] PATH_INFO: `/v3/projects/8cc067a77f4f4cb3927903c895afcb51/users/8fd2822c3a944a4ba930c76c54e25482/roles/114e6a16e63447d1924707ace5aace25` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:51.344079 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7290bf20-15ce-405b-b1de-64073c670260 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8cc067a77f4f4cb3927903c895afcb51, user_id=8fd2822c3a944a4ba930c76c54e25482, role_id=114e6a16e63447d1924707ace5aace25)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:51.347556 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7290bf20-15ce-405b-b1de-64073c670260 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:51.383910 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 844/1702] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:29:51 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:29:51.385172 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4909506d-50c6-49ce-80c3-60dbd27b9265 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:51.385624 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4909506d-50c6-49ce-80c3-60dbd27b9265 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:51.385901 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4909506d-50c6-49ce-80c3-60dbd27b9265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:51.418895 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-4909506d-50c6-49ce-80c3-60dbd27b9265 None None] MFA Rules not processed for user `2b90b57c542440448579b3aadda8b241`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:29:51.420167 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-4909506d-50c6-49ce-80c3-60dbd27b9265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:29:51.431475 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 859/1703] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:29:51 2022] PUT /identity/v3/projects/8cc067a77f4f4cb3927903c895afcb51/users/8fd2822c3a944a4ba930c76c54e25482/roles/114e6a16e63447d1924707ace5aace25 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:29:51.444460 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7290bf20-15ce-405b-b1de-64073c670260 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:51.449336 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-7290bf20-15ce-405b-b1de-64073c670260 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:29:51.561673 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 845/1704] 10.222.0.58 () {58 vars in 1009 bytes} [Tue May 17 19:29:51 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:29:51.564121 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d11d68b-65f4-4677-a825-7a9c26793fb3 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:51.564434 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d11d68b-65f4-4677-a825-7a9c26793fb3 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:51.564710 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d11d68b-65f4-4677-a825-7a9c26793fb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:51.615559 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-6d11d68b-65f4-4677-a825-7a9c26793fb3 None None] MFA Rules not processed for user `8fd2822c3a944a4ba930c76c54e25482`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:29:51.618937 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-6d11d68b-65f4-4677-a825-7a9c26793fb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:29:51.683622 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7290bf20-15ce-405b-b1de-64073c670260 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:51.691756 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d29fe1-3077-4317-a6cc-d102a67b6ab6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c1f878b0dc4d63a4f5dcb12fac1dc1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:51.692532 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d29fe1-3077-4317-a6cc-d102a67b6ab6 service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:51.692814 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d29fe1-3077-4317-a6cc-d102a67b6ab6 service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:51.693085 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d29fe1-3077-4317-a6cc-d102a67b6ab6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:51.794461 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 846/1705] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:29:51 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:29:51.799062 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d11d68b-65f4-4677-a825-7a9c26793fb3 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:51.802320 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-6d11d68b-65f4-4677-a825-7a9c26793fb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:29:51.855936 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d29fe1-3077-4317-a6cc-d102a67b6ab6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:51.857898 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d29fe1-3077-4317-a6cc-d102a67b6ab6 service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:51.906691 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 860/1706] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:29:51 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:29:51.907710 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d29fe1-3077-4317-a6cc-d102a67b6ab6 service nova] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:51.980390 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d11d68b-65f4-4677-a825-7a9c26793fb3 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:51.999696 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72f01a4-18ab-4fab-a7ca-0600d26efa8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:52.000811 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72f01a4-18ab-4fab-a7ca-0600d26efa8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:52.005130 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72f01a4-18ab-4fab-a7ca-0600d26efa8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:52.005130 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72f01a4-18ab-4fab-a7ca-0600d26efa8f admin admin] PATH_INFO: `/v3/projects/4425ca36d0fd4d779229d4003a0a607c` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:52.007354 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72f01a4-18ab-4fab-a7ca-0600d26efa8f admin admin] RBAC: Authorizing `identity:delete_project(project_id=4425ca36d0fd4d779229d4003a0a607c)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:52.014594 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72f01a4-18ab-4fab-a7ca-0600d26efa8f admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:52.205623 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-a72f01a4-18ab-4fab-a7ca-0600d26efa8f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4425ca36d0fd4d779229d4003a0a607c', 'request_id': 'req-a72f01a4-18ab-4fab-a7ca-0600d26efa8f'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:29:52.222437 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-a72f01a4-18ab-4fab-a7ca-0600d26efa8f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4425ca36d0fd4d779229d4003a0a607c', 'request_id': 'req-a72f01a4-18ab-4fab-a7ca-0600d26efa8f'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:29:52.237274 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-a72f01a4-18ab-4fab-a7ca-0600d26efa8f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4425ca36d0fd4d779229d4003a0a607c', 'request_id': 'req-a72f01a4-18ab-4fab-a7ca-0600d26efa8f'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:29:52.308687 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d29fe1-3077-4317-a6cc-d102a67b6ab6 service nova] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:52.315001 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-a72f01a4-18ab-4fab-a7ca-0600d26efa8f admin admin] The token cache is being invalidate because project 4425ca36d0fd4d779229d4003a0a607c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 17 19:29:52.315186 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-a72f01a4-18ab-4fab-a7ca-0600d26efa8f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:29:52.315997 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-a72f01a4-18ab-4fab-a7ca-0600d26efa8f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:29:52.317847 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 847/1707] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 17 19:29:51 2022] DELETE /identity/v3/projects/4425ca36d0fd4d779229d4003a0a607c => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:29:52.318631 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72f01a4-18ab-4fab-a7ca-0600d26efa8f admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:52.324498 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-a72f01a4-18ab-4fab-a7ca-0600d26efa8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:29:52.404244 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75abd8a-35ac-4d3b-b3d9-602603c37f15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:52.405388 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75abd8a-35ac-4d3b-b3d9-602603c37f15 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:52.405856 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75abd8a-35ac-4d3b-b3d9-602603c37f15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:52.406404 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75abd8a-35ac-4d3b-b3d9-602603c37f15 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:52.407518 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75abd8a-35ac-4d3b-b3d9-602603c37f15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:52.409519 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72f01a4-18ab-4fab-a7ca-0600d26efa8f admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:52.409846 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75abd8a-35ac-4d3b-b3d9-602603c37f15 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:52.414550 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07762c65-4987-47fc-a769-92bf003080c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:52.415763 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07762c65-4987-47fc-a769-92bf003080c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:52.415976 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07762c65-4987-47fc-a769-92bf003080c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:52.416128 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07762c65-4987-47fc-a769-92bf003080c7 admin admin] PATH_INFO: `/v3/projects/a1b94ef50c204b5e95f741ded1bcc77f` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:52.442477 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07762c65-4987-47fc-a769-92bf003080c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1b94ef50c204b5e95f741ded1bcc77f)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:52.461249 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07762c65-4987-47fc-a769-92bf003080c7 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:52.513595 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 861/1708] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 17 19:29:51 2022] POST /identity/v3/projects => generated 387 bytes in 606 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:29:52.514715 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75abd8a-35ac-4d3b-b3d9-602603c37f15 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:52.517566 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-f75abd8a-35ac-4d3b-b3d9-602603c37f15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:29:52.627791 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-07762c65-4987-47fc-a769-92bf003080c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1b94ef50c204b5e95f741ded1bcc77f', 'request_id': 'req-07762c65-4987-47fc-a769-92bf003080c7'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:29:52.653316 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-07762c65-4987-47fc-a769-92bf003080c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1b94ef50c204b5e95f741ded1bcc77f', 'request_id': 'req-07762c65-4987-47fc-a769-92bf003080c7'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:29:52.679923 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-07762c65-4987-47fc-a769-92bf003080c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1b94ef50c204b5e95f741ded1bcc77f', 'request_id': 'req-07762c65-4987-47fc-a769-92bf003080c7'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:29:52.702205 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75abd8a-35ac-4d3b-b3d9-602603c37f15 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:52.786020 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-07762c65-4987-47fc-a769-92bf003080c7 admin admin] The token cache is being invalidate because project a1b94ef50c204b5e95f741ded1bcc77f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 17 19:29:52.790055 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-07762c65-4987-47fc-a769-92bf003080c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:29:52.790055 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-07762c65-4987-47fc-a769-92bf003080c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:29:52.792481 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 848/1709] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 17 19:29:52 2022] DELETE /identity/v3/projects/a1b94ef50c204b5e95f741ded1bcc77f => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:29:52.795623 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07762c65-4987-47fc-a769-92bf003080c7 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:52.797225 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-07762c65-4987-47fc-a769-92bf003080c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:29:52.835396 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3a4902-464c-43d2-a083-828fad5929c5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0cb292002434d30bc81f370d120a2a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:52.837440 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3a4902-464c-43d2-a083-828fad5929c5 service swift] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:52.839127 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3a4902-464c-43d2-a083-828fad5929c5 service swift] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:52.839710 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3a4902-464c-43d2-a083-828fad5929c5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:52.845091 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-6d3a4902-464c-43d2-a083-828fad5929c5 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:29:52.955078 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07762c65-4987-47fc-a769-92bf003080c7 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:52.966990 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebf6298-73e1-46aa-b3bb-012d6432e70a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:52.968554 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebf6298-73e1-46aa-b3bb-012d6432e70a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:52.968909 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebf6298-73e1-46aa-b3bb-012d6432e70a admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:52.969253 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebf6298-73e1-46aa-b3bb-012d6432e70a admin admin] PATH_INFO: `/v3/users/1c873f2155834cdfb3870e232ee84697` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:52.973427 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebf6298-73e1-46aa-b3bb-012d6432e70a admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c873f2155834cdfb3870e232ee84697)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:52.976897 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebf6298-73e1-46aa-b3bb-012d6432e70a admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:53.008474 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3a4902-464c-43d2-a083-828fad5929c5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:53.010512 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3a4902-464c-43d2-a083-828fad5929c5 service swift] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:53.026298 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 862/1710] 10.222.0.58 () {60 vars in 1384 bytes} [Tue May 17 19:29:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 512 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 17 19:29:53.028589 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f627a8b-f29d-447d-bfce-85f28741f2b2 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:53.029030 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f627a8b-f29d-447d-bfce-85f28741f2b2 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:53.029367 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f627a8b-f29d-447d-bfce-85f28741f2b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:53.090832 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-4f627a8b-f29d-447d-bfce-85f28741f2b2 None None] MFA Rules not processed for user `6545bb5d5eb24c199b5d89091b2c1545`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:29:53.094724 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-4f627a8b-f29d-447d-bfce-85f28741f2b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:29:53.212130 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 863/1711] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:29:53 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:29:53.213753 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f627a8b-f29d-447d-bfce-85f28741f2b2 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:53.217008 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-4f627a8b-f29d-447d-bfce-85f28741f2b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:29:53.270475 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f627a8b-f29d-447d-bfce-85f28741f2b2 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:53.277324 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-aebf6298-73e1-46aa-b3bb-012d6432e70a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c873f2155834cdfb3870e232ee84697'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:29:53.290804 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9e60cc-5106-43db-ae8a-56ccc1dcd1cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:53.291916 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9e60cc-5106-43db-ae8a-56ccc1dcd1cb admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:53.292211 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9e60cc-5106-43db-ae8a-56ccc1dcd1cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:53.292483 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9e60cc-5106-43db-ae8a-56ccc1dcd1cb admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:53.293174 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9e60cc-5106-43db-ae8a-56ccc1dcd1cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:53.298462 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9e60cc-5106-43db-ae8a-56ccc1dcd1cb admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:53.301641 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-aebf6298-73e1-46aa-b3bb-012d6432e70a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c873f2155834cdfb3870e232ee84697'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:29:53.337171 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-aebf6298-73e1-46aa-b3bb-012d6432e70a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c873f2155834cdfb3870e232ee84697'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:29:53.339172 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-aebf6298-73e1-46aa-b3bb-012d6432e70a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c873f2155834cdfb3870e232ee84697'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:29:53.341052 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-aebf6298-73e1-46aa-b3bb-012d6432e70a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c873f2155834cdfb3870e232ee84697'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:29:53.431478 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 849/1712] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 17 19:29:52 2022] DELETE /identity/v3/users/1c873f2155834cdfb3870e232ee84697 => generated 0 bytes in 638 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:29:53.432777 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebf6298-73e1-46aa-b3bb-012d6432e70a admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:53.434481 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-aebf6298-73e1-46aa-b3bb-012d6432e70a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:29:53.435665 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 864/1713] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 17 19:29:53 2022] POST /identity/v3/users => generated 294 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:29:53.437071 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9e60cc-5106-43db-ae8a-56ccc1dcd1cb admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:53.442305 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-1b9e60cc-5106-43db-ae8a-56ccc1dcd1cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:29:53.607688 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebf6298-73e1-46aa-b3bb-012d6432e70a admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:53.637704 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c42b39-34e1-4e9b-8ec9-863fbd172b6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:53.638736 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c42b39-34e1-4e9b-8ec9-863fbd172b6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:53.661292 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c42b39-34e1-4e9b-8ec9-863fbd172b6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:53.663268 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c42b39-34e1-4e9b-8ec9-863fbd172b6a admin admin] PATH_INFO: `/v3/users/a63a1d17dc364eed8bb25ed64745b04c` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:53.669023 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c42b39-34e1-4e9b-8ec9-863fbd172b6a admin admin] RBAC: Authorizing `identity:delete_user(user_id=a63a1d17dc364eed8bb25ed64745b04c)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:53.671911 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c42b39-34e1-4e9b-8ec9-863fbd172b6a admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:53.702080 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9e60cc-5106-43db-ae8a-56ccc1dcd1cb admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:53.829122 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff81186-5590-4803-95fd-4e5208bc1d96 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c1f878b0dc4d63a4f5dcb12fac1dc1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:53.830209 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff81186-5590-4803-95fd-4e5208bc1d96 service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:53.830628 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff81186-5590-4803-95fd-4e5208bc1d96 service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:53.831073 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff81186-5590-4803-95fd-4e5208bc1d96 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:53.833874 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-7ff81186-5590-4803-95fd-4e5208bc1d96 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:29:53.929513 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-d8c42b39-34e1-4e9b-8ec9-863fbd172b6a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a63a1d17dc364eed8bb25ed64745b04c'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:29:53.941648 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-d8c42b39-34e1-4e9b-8ec9-863fbd172b6a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a63a1d17dc364eed8bb25ed64745b04c'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:29:54.008565 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff81186-5590-4803-95fd-4e5208bc1d96 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:54.008565 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff81186-5590-4803-95fd-4e5208bc1d96 service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:54.008737 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-d8c42b39-34e1-4e9b-8ec9-863fbd172b6a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a63a1d17dc364eed8bb25ed64745b04c'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:29:54.008737 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-d8c42b39-34e1-4e9b-8ec9-863fbd172b6a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a63a1d17dc364eed8bb25ed64745b04c'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:29:54.008737 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-d8c42b39-34e1-4e9b-8ec9-863fbd172b6a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a63a1d17dc364eed8bb25ed64745b04c'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:29:54.008924 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 865/1714] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:29:53 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 571 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:29:54.011157 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747e63d0-f6d7-4162-98d6-05fe9c3f7071 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:54.011654 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747e63d0-f6d7-4162-98d6-05fe9c3f7071 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:54.011934 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747e63d0-f6d7-4162-98d6-05fe9c3f7071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:54.064805 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-747e63d0-f6d7-4162-98d6-05fe9c3f7071 None None] MFA Rules not processed for user `1ac193d15f48463c98ff1d97770ed794`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:29:54.064805 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-747e63d0-f6d7-4162-98d6-05fe9c3f7071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:29:54.198545 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 866/1715] 10.222.0.69 () {58 vars in 1015 bytes} [Tue May 17 19:29:54 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:29:54.200409 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747e63d0-f6d7-4162-98d6-05fe9c3f7071 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:54.205185 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-747e63d0-f6d7-4162-98d6-05fe9c3f7071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:29:54.224612 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747e63d0-f6d7-4162-98d6-05fe9c3f7071 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:54.241579 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196c6475-d46f-46c6-9f5e-fdc5a1bbbecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:54.242478 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196c6475-d46f-46c6-9f5e-fdc5a1bbbecf admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:54.244164 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196c6475-d46f-46c6-9f5e-fdc5a1bbbecf admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:54.244601 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196c6475-d46f-46c6-9f5e-fdc5a1bbbecf admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:54.245999 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196c6475-d46f-46c6-9f5e-fdc5a1bbbecf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:54.250703 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196c6475-d46f-46c6-9f5e-fdc5a1bbbecf admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:54.271392 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 867/1716] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:29:54 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:29:54.273869 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196c6475-d46f-46c6-9f5e-fdc5a1bbbecf admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:54.302027 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196c6475-d46f-46c6-9f5e-fdc5a1bbbecf admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:54.318527 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cce3b5a-a0da-4469-bfd4-cd837d43cdc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:54.319364 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cce3b5a-a0da-4469-bfd4-cd837d43cdc5 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:54.319760 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cce3b5a-a0da-4469-bfd4-cd837d43cdc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:54.320305 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cce3b5a-a0da-4469-bfd4-cd837d43cdc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:54.321135 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cce3b5a-a0da-4469-bfd4-cd837d43cdc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:54.323020 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cce3b5a-a0da-4469-bfd4-cd837d43cdc5 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:54.353140 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 868/1717] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:29:54 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:29:54.357195 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cce3b5a-a0da-4469-bfd4-cd837d43cdc5 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:54.393656 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 850/1718] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 17 19:29:53 2022] DELETE /identity/v3/users/a63a1d17dc364eed8bb25ed64745b04c => generated 0 bytes in 962 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:29:54.415303 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cce3b5a-a0da-4469-bfd4-cd837d43cdc5 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:54.515122 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c42b39-34e1-4e9b-8ec9-863fbd172b6a admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:54.516840 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-d8c42b39-34e1-4e9b-8ec9-863fbd172b6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:29:54.650845 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684d3f17-5034-4f3f-bca9-a7e34ee3bbc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:54.651639 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684d3f17-5034-4f3f-bca9-a7e34ee3bbc7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:54.652049 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684d3f17-5034-4f3f-bca9-a7e34ee3bbc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:54.652409 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684d3f17-5034-4f3f-bca9-a7e34ee3bbc7 admin admin] PATH_INFO: `/v3/projects/fa693b3ca120463a8d990d2406524331/users/fa7751acd00b4a468082ad9c77626aa6/roles/114e6a16e63447d1924707ace5aace25` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:54.684280 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684d3f17-5034-4f3f-bca9-a7e34ee3bbc7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa693b3ca120463a8d990d2406524331, user_id=fa7751acd00b4a468082ad9c77626aa6, role_id=114e6a16e63447d1924707ace5aace25)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:54.687292 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684d3f17-5034-4f3f-bca9-a7e34ee3bbc7 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:54.699286 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c42b39-34e1-4e9b-8ec9-863fbd172b6a admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:54.705285 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21535cbf-5a4f-417d-9b80-0df0c48e0c68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:54.706003 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21535cbf-5a4f-417d-9b80-0df0c48e0c68 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:54.706282 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21535cbf-5a4f-417d-9b80-0df0c48e0c68 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:54.706571 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21535cbf-5a4f-417d-9b80-0df0c48e0c68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:54.718201 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21535cbf-5a4f-417d-9b80-0df0c48e0c68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:54.721091 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21535cbf-5a4f-417d-9b80-0df0c48e0c68 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:54.745709 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 851/1719] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:29:54 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:29:54.788088 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21535cbf-5a4f-417d-9b80-0df0c48e0c68 service neutron] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:54.791875 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-21535cbf-5a4f-417d-9b80-0df0c48e0c68 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:29:54.853532 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 869/1720] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:29:54 2022] PUT /identity/v3/projects/fa693b3ca120463a8d990d2406524331/users/fa7751acd00b4a468082ad9c77626aa6/roles/114e6a16e63447d1924707ace5aace25 => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 17 19:29:54.856242 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabd9900-a850-477e-b874-e483ea525f03 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:54.858607 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabd9900-a850-477e-b874-e483ea525f03 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:54.859051 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabd9900-a850-477e-b874-e483ea525f03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:54.885389 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21535cbf-5a4f-417d-9b80-0df0c48e0c68 service neutron] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:54.975804 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-cabd9900-a850-477e-b874-e483ea525f03 None None] MFA Rules not processed for user `b171cf633e6e437fb509857b0a2962e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:29:54.979117 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-cabd9900-a850-477e-b874-e483ea525f03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:29:55.023441 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138f5aa1-bc76-4eda-ba8b-d887478ec9da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:55.026579 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138f5aa1-bc76-4eda-ba8b-d887478ec9da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:55.026579 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138f5aa1-bc76-4eda-ba8b-d887478ec9da admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:55.026782 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138f5aa1-bc76-4eda-ba8b-d887478ec9da admin admin] PATH_INFO: `/v3/projects/ea29931dd9f442c2aa79529d6cb18e75` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:55.031327 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138f5aa1-bc76-4eda-ba8b-d887478ec9da admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea29931dd9f442c2aa79529d6cb18e75)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:55.036405 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138f5aa1-bc76-4eda-ba8b-d887478ec9da admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:55.206967 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 870/1721] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:29:54 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 352 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:29:55.215424 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0a6dba-0ce7-42a7-affb-eba376101728 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:55.216725 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0a6dba-0ce7-42a7-affb-eba376101728 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:55.218205 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0a6dba-0ce7-42a7-affb-eba376101728 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:55.293216 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-138f5aa1-bc76-4eda-ba8b-d887478ec9da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea29931dd9f442c2aa79529d6cb18e75', 'request_id': 'req-138f5aa1-bc76-4eda-ba8b-d887478ec9da'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:29:55.329088 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-6e0a6dba-0ce7-42a7-affb-eba376101728 None None] MFA Rules not processed for user `fa7751acd00b4a468082ad9c77626aa6`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:29:55.332447 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-6e0a6dba-0ce7-42a7-affb-eba376101728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:29:55.360262 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-138f5aa1-bc76-4eda-ba8b-d887478ec9da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea29931dd9f442c2aa79529d6cb18e75', 'request_id': 'req-138f5aa1-bc76-4eda-ba8b-d887478ec9da'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:29:55.383252 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-138f5aa1-bc76-4eda-ba8b-d887478ec9da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea29931dd9f442c2aa79529d6cb18e75', 'request_id': 'req-138f5aa1-bc76-4eda-ba8b-d887478ec9da'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:29:55.498113 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-138f5aa1-bc76-4eda-ba8b-d887478ec9da admin admin] The token cache is being invalidate because project ea29931dd9f442c2aa79529d6cb18e75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 17 19:29:55.499104 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-138f5aa1-bc76-4eda-ba8b-d887478ec9da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:29:55.503805 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-138f5aa1-bc76-4eda-ba8b-d887478ec9da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:29:55.503805 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 852/1722] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 17 19:29:54 2022] DELETE /identity/v3/projects/ea29931dd9f442c2aa79529d6cb18e75 => generated 0 bytes in 716 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:29:55.513727 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15294902-13e5-4932-bd44-833315e9deec None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:55.514153 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15294902-13e5-4932-bd44-833315e9deec None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:55.515002 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15294902-13e5-4932-bd44-833315e9deec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:55.562878 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 871/1723] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:29:55 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 351 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:29:55.576411 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0a6dba-0ce7-42a7-affb-eba376101728 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:55.582537 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-6e0a6dba-0ce7-42a7-affb-eba376101728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:29:55.592197 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-15294902-13e5-4932-bd44-833315e9deec None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:29:55.606125 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-15294902-13e5-4932-bd44-833315e9deec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:29:55.856102 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0a6dba-0ce7-42a7-affb-eba376101728 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:55.883776 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dc9a62-da80-4a87-a5c6-272f308a6261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:55.890609 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dc9a62-da80-4a87-a5c6-272f308a6261 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:55.891066 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 853/1724] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:29:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 384 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:29:55.891736 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dc9a62-da80-4a87-a5c6-272f308a6261 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:55.892032 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dc9a62-da80-4a87-a5c6-272f308a6261 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:55.898131 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32161972-3beb-4874-bd5c-cf042eed1047 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:55.898425 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32161972-3beb-4874-bd5c-cf042eed1047 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:55.899151 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dc9a62-da80-4a87-a5c6-272f308a6261 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:55.902910 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dc9a62-da80-4a87-a5c6-272f308a6261 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:55.906026 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32161972-3beb-4874-bd5c-cf042eed1047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:56.000343 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-32161972-3beb-4874-bd5c-cf042eed1047 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:29:56.006354 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-32161972-3beb-4874-bd5c-cf042eed1047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:29:56.055388 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 872/1725] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 17 19:29:55 2022] POST /identity/v3/projects => generated 387 bytes in 481 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:29:56.058179 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 854/1726] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:29:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:29:56.063171 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dc9a62-da80-4a87-a5c6-272f308a6261 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:56.064090 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32161972-3beb-4874-bd5c-cf042eed1047 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:56.105562 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32161972-3beb-4874-bd5c-cf042eed1047 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:56.399350 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dc9a62-da80-4a87-a5c6-272f308a6261 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:56.430492 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5c03ad-3aee-46e9-b262-84abbdab05fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:56.431181 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5c03ad-3aee-46e9-b262-84abbdab05fa admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:56.431466 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5c03ad-3aee-46e9-b262-84abbdab05fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:56.437016 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5c03ad-3aee-46e9-b262-84abbdab05fa admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:56.437809 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5c03ad-3aee-46e9-b262-84abbdab05fa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:29:56.438121 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5c03ad-3aee-46e9-b262-84abbdab05fa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:56.439813 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5c03ad-3aee-46e9-b262-84abbdab05fa admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:56.449042 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ce8d82-bce5-446a-b012-60efdf26c80e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:56.449756 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ce8d82-bce5-446a-b012-60efdf26c80e admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:56.450039 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ce8d82-bce5-446a-b012-60efdf26c80e admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:56.450315 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ce8d82-bce5-446a-b012-60efdf26c80e admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:56.456293 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ce8d82-bce5-446a-b012-60efdf26c80e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:56.458290 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ce8d82-bce5-446a-b012-60efdf26c80e admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:56.503570 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 873/1727] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 17 19:29:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 433 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:29:56.504650 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5c03ad-3aee-46e9-b262-84abbdab05fa admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:56.535124 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5c03ad-3aee-46e9-b262-84abbdab05fa admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:56.543403 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8470a397-411a-4353-a3cb-cbf3b19aae8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:56.547441 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8470a397-411a-4353-a3cb-cbf3b19aae8c admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:56.547441 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8470a397-411a-4353-a3cb-cbf3b19aae8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:56.547441 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8470a397-411a-4353-a3cb-cbf3b19aae8c admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:56.547441 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8470a397-411a-4353-a3cb-cbf3b19aae8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:56.548401 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8470a397-411a-4353-a3cb-cbf3b19aae8c admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:56.614154 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 855/1728] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 17 19:29:56 2022] POST /identity/v3/users => generated 300 bytes in 553 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:29:56.617713 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ce8d82-bce5-446a-b012-60efdf26c80e admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:56.651976 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ce8d82-bce5-446a-b012-60efdf26c80e admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:56.687992 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 874/1729] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 17 19:29:56 2022] POST /identity/v3/projects => generated 369 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:29:56.688906 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8470a397-411a-4353-a3cb-cbf3b19aae8c admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:56.690007 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7520d1ad-8a3b-4e3a-a98a-d4f3209df490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:56.694085 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7520d1ad-8a3b-4e3a-a98a-d4f3209df490 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:56.694917 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7520d1ad-8a3b-4e3a-a98a-d4f3209df490 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:56.697500 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7520d1ad-8a3b-4e3a-a98a-d4f3209df490 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:56.700659 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7520d1ad-8a3b-4e3a-a98a-d4f3209df490 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:56.703475 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7520d1ad-8a3b-4e3a-a98a-d4f3209df490 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:56.761635 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 856/1730] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:29:56 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:29:56.765127 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7520d1ad-8a3b-4e3a-a98a-d4f3209df490 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:56.812898 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7520d1ad-8a3b-4e3a-a98a-d4f3209df490 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:56.938493 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8470a397-411a-4353-a3cb-cbf3b19aae8c admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:56.969243 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0176d503-0705-4b71-a822-46f010b49b7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:56.970108 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0176d503-0705-4b71-a822-46f010b49b7b admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:56.970403 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0176d503-0705-4b71-a822-46f010b49b7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:56.974488 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0176d503-0705-4b71-a822-46f010b49b7b admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:56.974860 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0176d503-0705-4b71-a822-46f010b49b7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:56.980329 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0176d503-0705-4b71-a822-46f010b49b7b admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:57.015449 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9912fd4b-15f5-4973-9d62-30c0dbade97b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:57.019313 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9912fd4b-15f5-4973-9d62-30c0dbade97b admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:57.019541 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9912fd4b-15f5-4973-9d62-30c0dbade97b admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:57.019833 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9912fd4b-15f5-4973-9d62-30c0dbade97b admin admin] PATH_INFO: `/v3/projects/4884f06209034b2c85462aab113ad3b9/users/43256ad6f4da4d20bc3063a135eaf8d4/roles/60af5088793e4379b88821a86552a344` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:57.064657 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9912fd4b-15f5-4973-9d62-30c0dbade97b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4884f06209034b2c85462aab113ad3b9, user_id=43256ad6f4da4d20bc3063a135eaf8d4, role_id=60af5088793e4379b88821a86552a344)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:57.069981 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9912fd4b-15f5-4973-9d62-30c0dbade97b admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:57.117826 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 875/1731] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 17 19:29:56 2022] POST /identity/v3/users => generated 285 bytes in 428 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:29:57.117826 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0176d503-0705-4b71-a822-46f010b49b7b admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:57.120810 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-0176d503-0705-4b71-a822-46f010b49b7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:29:57.168580 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0176d503-0705-4b71-a822-46f010b49b7b admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:57.179394 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 857/1732] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:29:56 2022] PUT /identity/v3/projects/4884f06209034b2c85462aab113ad3b9/users/43256ad6f4da4d20bc3063a135eaf8d4/roles/60af5088793e4379b88821a86552a344 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:29:57.180301 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9912fd4b-15f5-4973-9d62-30c0dbade97b admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:57.184342 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-9912fd4b-15f5-4973-9d62-30c0dbade97b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:29:57.195159 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a1d385-4fff-42f3-b0a3-043a3e97f40b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:57.196212 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a1d385-4fff-42f3-b0a3-043a3e97f40b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:57.196508 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a1d385-4fff-42f3-b0a3-043a3e97f40b admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:57.197806 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a1d385-4fff-42f3-b0a3-043a3e97f40b admin admin] PATH_INFO: `/v3/projects/a5a5bfa5186f41698644470d9ecc8943` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:57.203734 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a1d385-4fff-42f3-b0a3-043a3e97f40b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5a5bfa5186f41698644470d9ecc8943)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:57.209791 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a1d385-4fff-42f3-b0a3-043a3e97f40b admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:57.357002 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9912fd4b-15f5-4973-9d62-30c0dbade97b admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:57.383320 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d78077d-b012-48b9-aabd-c199c502c4bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:57.384739 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d78077d-b012-48b9-aabd-c199c502c4bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:57.385021 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d78077d-b012-48b9-aabd-c199c502c4bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:57.388062 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d78077d-b012-48b9-aabd-c199c502c4bc admin admin] PATH_INFO: `/v3/users/33b957d0521c4fcd98f844020b7becd1` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:57.393719 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d78077d-b012-48b9-aabd-c199c502c4bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=33b957d0521c4fcd98f844020b7becd1)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:57.403322 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d78077d-b012-48b9-aabd-c199c502c4bc admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:57.414760 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-28a1d385-4fff-42f3-b0a3-043a3e97f40b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5a5bfa5186f41698644470d9ecc8943', 'request_id': 'req-28a1d385-4fff-42f3-b0a3-043a3e97f40b'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:29:57.442193 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-28a1d385-4fff-42f3-b0a3-043a3e97f40b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5a5bfa5186f41698644470d9ecc8943', 'request_id': 'req-28a1d385-4fff-42f3-b0a3-043a3e97f40b'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:29:57.597271 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-28a1d385-4fff-42f3-b0a3-043a3e97f40b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5a5bfa5186f41698644470d9ecc8943', 'request_id': 'req-28a1d385-4fff-42f3-b0a3-043a3e97f40b'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:29:57.672245 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-28a1d385-4fff-42f3-b0a3-043a3e97f40b admin admin] The token cache is being invalidate because project a5a5bfa5186f41698644470d9ecc8943 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 17 19:29:57.682229 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-28a1d385-4fff-42f3-b0a3-043a3e97f40b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:29:57.686797 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-28a1d385-4fff-42f3-b0a3-043a3e97f40b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:29:57.708033 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 876/1733] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 17 19:29:57 2022] DELETE /identity/v3/projects/a5a5bfa5186f41698644470d9ecc8943 => generated 0 bytes in 588 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:29:57.710997 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a1d385-4fff-42f3-b0a3-043a3e97f40b admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:57.714382 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-28a1d385-4fff-42f3-b0a3-043a3e97f40b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:29:57.772738 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-2d78077d-b012-48b9-aabd-c199c502c4bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33b957d0521c4fcd98f844020b7becd1'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:29:57.801782 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-2d78077d-b012-48b9-aabd-c199c502c4bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33b957d0521c4fcd98f844020b7becd1'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:29:57.926814 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-2d78077d-b012-48b9-aabd-c199c502c4bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33b957d0521c4fcd98f844020b7becd1'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:29:57.928853 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-2d78077d-b012-48b9-aabd-c199c502c4bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33b957d0521c4fcd98f844020b7becd1'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:29:57.932494 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-2d78077d-b012-48b9-aabd-c199c502c4bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33b957d0521c4fcd98f844020b7becd1'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:29:58.062790 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 858/1734] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 17 19:29:57 2022] DELETE /identity/v3/users/33b957d0521c4fcd98f844020b7becd1 => generated 0 bytes in 883 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:29:58.064537 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d78077d-b012-48b9-aabd-c199c502c4bc admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:58.068179 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-2d78077d-b012-48b9-aabd-c199c502c4bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:29:58.088554 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a1d385-4fff-42f3-b0a3-043a3e97f40b admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:58.343015 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a12dfac-31b6-4534-be75-471c7a04ca63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:58.343015 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a12dfac-31b6-4534-be75-471c7a04ca63 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:58.343015 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a12dfac-31b6-4534-be75-471c7a04ca63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:58.343015 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a12dfac-31b6-4534-be75-471c7a04ca63 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:58.343386 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a12dfac-31b6-4534-be75-471c7a04ca63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:58.345872 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a12dfac-31b6-4534-be75-471c7a04ca63 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:58.400875 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d78077d-b012-48b9-aabd-c199c502c4bc admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:58.405318 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 877/1735] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:29:57 2022] GET /identity/v3/roles => generated 1420 bytes in 695 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:29:58.405318 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8d06f7-b6a0-4abf-8cfb-ba5bb97b3dba None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:58.417211 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8d06f7-b6a0-4abf-8cfb-ba5bb97b3dba None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:58.417961 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d075c45-3fd6-4fe0-991f-f60e05933ed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:58.419134 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d075c45-3fd6-4fe0-991f-f60e05933ed0 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:58.419419 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d075c45-3fd6-4fe0-991f-f60e05933ed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:58.419561 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8d06f7-b6a0-4abf-8cfb-ba5bb97b3dba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:58.419809 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d075c45-3fd6-4fe0-991f-f60e05933ed0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:58.421530 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d075c45-3fd6-4fe0-991f-f60e05933ed0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:58.424053 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d075c45-3fd6-4fe0-991f-f60e05933ed0 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:58.438865 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 859/1736] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:29:58 2022] GET /identity/v3/roles => generated 1420 bytes in 374 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:29:58.440608 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17535956-99fb-4613-8dbb-807f306c322a None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:58.440898 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17535956-99fb-4613-8dbb-807f306c322a None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:58.441183 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17535956-99fb-4613-8dbb-807f306c322a None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:58.453607 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 860/1737] 10.222.0.58 () {54 vars in 921 bytes} [Tue May 17 19:29:58 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:29:58.464723 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5618d41c-180a-4c8b-aa1f-3b47cffe2713 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:58.465166 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5618d41c-180a-4c8b-aa1f-3b47cffe2713 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:58.465453 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5618d41c-180a-4c8b-aa1f-3b47cffe2713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:58.512880 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-dc8d06f7-b6a0-4abf-8cfb-ba5bb97b3dba None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:29:58.514539 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-5618d41c-180a-4c8b-aa1f-3b47cffe2713 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:29:58.515710 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-5618d41c-180a-4c8b-aa1f-3b47cffe2713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:29:58.523866 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-dc8d06f7-b6a0-4abf-8cfb-ba5bb97b3dba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:29:58.554863 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 861/1738] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:29:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:29:58.556116 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5618d41c-180a-4c8b-aa1f-3b47cffe2713 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:58.560106 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-5618d41c-180a-4c8b-aa1f-3b47cffe2713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:29:58.567382 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 878/1739] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:29:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:29:58.567382 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8d06f7-b6a0-4abf-8cfb-ba5bb97b3dba None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:58.590692 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5618d41c-180a-4c8b-aa1f-3b47cffe2713 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:58.599842 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8d06f7-b6a0-4abf-8cfb-ba5bb97b3dba None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:58.602176 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300f5180-4fca-4d5e-8ef8-b5acc33acdc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:58.605289 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300f5180-4fca-4d5e-8ef8-b5acc33acdc5 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:58.609755 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300f5180-4fca-4d5e-8ef8-b5acc33acdc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:58.609755 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300f5180-4fca-4d5e-8ef8-b5acc33acdc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:58.609755 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300f5180-4fca-4d5e-8ef8-b5acc33acdc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:58.614237 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300f5180-4fca-4d5e-8ef8-b5acc33acdc5 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:58.624264 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8caeb7c-c74e-4147-9782-f31db36af28e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:58.626159 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8caeb7c-c74e-4147-9782-f31db36af28e admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:58.626445 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8caeb7c-c74e-4147-9782-f31db36af28e admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:58.626717 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8caeb7c-c74e-4147-9782-f31db36af28e admin admin] PATH_INFO: `/v3/projects/4884f06209034b2c85462aab113ad3b9/users/43256ad6f4da4d20bc3063a135eaf8d4/roles/60af5088793e4379b88821a86552a344` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:58.631362 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8caeb7c-c74e-4147-9782-f31db36af28e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4884f06209034b2c85462aab113ad3b9, user_id=43256ad6f4da4d20bc3063a135eaf8d4, role_id=60af5088793e4379b88821a86552a344)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:58.644243 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 862/1740] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:29:58 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:29:58.644452 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8caeb7c-c74e-4147-9782-f31db36af28e admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:58.646262 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f958d35-00e7-44b1-bcd7-b9625bbd5337 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:58.647843 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f958d35-00e7-44b1-bcd7-b9625bbd5337 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:58.651113 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f958d35-00e7-44b1-bcd7-b9625bbd5337 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:58.678387 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 879/1741] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:29:58 2022] PUT /identity/v3/projects/4884f06209034b2c85462aab113ad3b9/users/43256ad6f4da4d20bc3063a135eaf8d4/roles/60af5088793e4379b88821a86552a344 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:29:58.678387 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082f1ce3-62a8-4784-a566-cedd5baae4e3 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:58.678387 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082f1ce3-62a8-4784-a566-cedd5baae4e3 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:58.678387 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082f1ce3-62a8-4784-a566-cedd5baae4e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:58.726932 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-2f958d35-00e7-44b1-bcd7-b9625bbd5337 None None] MFA Rules not processed for user `65c1f878b0dc4d63a4f5dcb12fac1dc1`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:29:58.732461 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-082f1ce3-62a8-4784-a566-cedd5baae4e3 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:29:58.734458 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-082f1ce3-62a8-4784-a566-cedd5baae4e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:29:58.737444 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-2f958d35-00e7-44b1-bcd7-b9625bbd5337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:29:58.949894 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 880/1742] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:29:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:29:58.950656 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082f1ce3-62a8-4784-a566-cedd5baae4e3 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:58.990846 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082f1ce3-62a8-4784-a566-cedd5baae4e3 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:59.008518 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9770701-3e0a-43c5-a785-0a67c5574b2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:59.008518 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9770701-3e0a-43c5-a785-0a67c5574b2e admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:59.008518 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9770701-3e0a-43c5-a785-0a67c5574b2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:59.008518 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9770701-3e0a-43c5-a785-0a67c5574b2e admin admin] PATH_INFO: `/v3/projects/1321d17bd52e406bb671194b9fab5173/users/b04c599f7a054d4b91aba25701a3ed5f/roles/114e6a16e63447d1924707ace5aace25` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:59.044515 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 863/1743] 10.222.0.58 () {58 vars in 1018 bytes} [Tue May 17 19:29:58 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 399 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:29:59.046812 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3e27e1-817e-4639-9171-4158d7b101e9 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:59.047218 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3e27e1-817e-4639-9171-4158d7b101e9 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:59.047368 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3e27e1-817e-4639-9171-4158d7b101e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:59.069915 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9770701-3e0a-43c5-a785-0a67c5574b2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1321d17bd52e406bb671194b9fab5173, user_id=b04c599f7a054d4b91aba25701a3ed5f, role_id=114e6a16e63447d1924707ace5aace25)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:59.081527 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9770701-3e0a-43c5-a785-0a67c5574b2e admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:59.157483 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-ab3e27e1-817e-4639-9171-4158d7b101e9 None None] MFA Rules not processed for user `43256ad6f4da4d20bc3063a135eaf8d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:29:59.158866 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-ab3e27e1-817e-4639-9171-4158d7b101e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:29:59.178510 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 881/1744] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:29:58 2022] PUT /identity/v3/projects/1321d17bd52e406bb671194b9fab5173/users/b04c599f7a054d4b91aba25701a3ed5f/roles/114e6a16e63447d1924707ace5aace25 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:29:59.180480 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9770701-3e0a-43c5-a785-0a67c5574b2e admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:59.183539 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a9770701-3e0a-43c5-a785-0a67c5574b2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:29:59.325384 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9770701-3e0a-43c5-a785-0a67c5574b2e admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:59.347123 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebd8036-cf32-4455-a62f-7df4a7e1640b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:59.347908 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebd8036-cf32-4455-a62f-7df4a7e1640b service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:59.348383 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebd8036-cf32-4455-a62f-7df4a7e1640b service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:59.348727 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebd8036-cf32-4455-a62f-7df4a7e1640b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:59.366079 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 864/1745] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:29:59 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:29:59.367449 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3e27e1-817e-4639-9171-4158d7b101e9 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:59.411232 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3e27e1-817e-4639-9171-4158d7b101e9 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:59.517525 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebd8036-cf32-4455-a62f-7df4a7e1640b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:59.519975 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebd8036-cf32-4455-a62f-7df4a7e1640b service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:59.542969 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 882/1746] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:29:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:29:59.544775 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a3c0a3-86f6-4320-957a-b43ca0bf4cd3 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:59.546139 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a3c0a3-86f6-4320-957a-b43ca0bf4cd3 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:59.546866 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a3c0a3-86f6-4320-957a-b43ca0bf4cd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:59.595981 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-79a3c0a3-86f6-4320-957a-b43ca0bf4cd3 None None] MFA Rules not processed for user `b04c599f7a054d4b91aba25701a3ed5f`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:29:59.598000 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-79a3c0a3-86f6-4320-957a-b43ca0bf4cd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:29:59.600375 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c45525-ed41-4ad6-b39d-242894dfe162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:59.601079 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c45525-ed41-4ad6-b39d-242894dfe162 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:59.604441 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c45525-ed41-4ad6-b39d-242894dfe162 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:59.604441 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c45525-ed41-4ad6-b39d-242894dfe162 admin admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:59.604441 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c45525-ed41-4ad6-b39d-242894dfe162 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:29:59.604441 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c45525-ed41-4ad6-b39d-242894dfe162 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:59.604441 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c45525-ed41-4ad6-b39d-242894dfe162 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:59.663245 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 865/1747] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 17 19:29:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:29:59.664420 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c45525-ed41-4ad6-b39d-242894dfe162 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:59.682956 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-f2c45525-ed41-4ad6-b39d-242894dfe162 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:29:59.760411 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 883/1748] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:29:59 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:29:59.762073 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a3c0a3-86f6-4320-957a-b43ca0bf4cd3 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:59.766076 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-79a3c0a3-86f6-4320-957a-b43ca0bf4cd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:29:59.825195 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c45525-ed41-4ad6-b39d-242894dfe162 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:59.832660 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3066ff5-3238-4e26-9e10-e97fd4596360 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c1f878b0dc4d63a4f5dcb12fac1dc1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:59.836509 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3066ff5-3238-4e26-9e10-e97fd4596360 service nova] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:59.836509 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3066ff5-3238-4e26-9e10-e97fd4596360 service nova] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:59.836509 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3066ff5-3238-4e26-9e10-e97fd4596360 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:59.846073 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3066ff5-3238-4e26-9e10-e97fd4596360 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:59.848250 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3066ff5-3238-4e26-9e10-e97fd4596360 service nova] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:29:59.861993 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 866/1749] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:29:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:29:59.862896 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3066ff5-3238-4e26-9e10-e97fd4596360 service nova] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:29:59.875007 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3066ff5-3238-4e26-9e10-e97fd4596360 service nova] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:29:59.887182 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4f3fc5-76e8-40b0-b40c-56810687dc77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:29:59.887182 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4f3fc5-76e8-40b0-b40c-56810687dc77 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:29:59.887182 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4f3fc5-76e8-40b0-b40c-56810687dc77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:29:59.887182 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4f3fc5-76e8-40b0-b40c-56810687dc77 admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:29:59.887584 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4f3fc5-76e8-40b0-b40c-56810687dc77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:29:59.887884 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4f3fc5-76e8-40b0-b40c-56810687dc77 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:00.019539 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a3c0a3-86f6-4320-957a-b43ca0bf4cd3 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:00.034166 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540070bb-1dee-4012-93f6-758ac5f73d6c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd05409b38ae346458abfe41ebda1c59e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:00.034836 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540070bb-1dee-4012-93f6-758ac5f73d6c service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:00.035115 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540070bb-1dee-4012-93f6-758ac5f73d6c service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:00.035388 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540070bb-1dee-4012-93f6-758ac5f73d6c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:00.094462 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 867/1750] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 17 19:29:59 2022] POST /identity/v3/projects => generated 381 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:30:00.098709 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4f3fc5-76e8-40b0-b40c-56810687dc77 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:00.307785 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540070bb-1dee-4012-93f6-758ac5f73d6c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:00.309277 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540070bb-1dee-4012-93f6-758ac5f73d6c service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:00.323593 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4f3fc5-76e8-40b0-b40c-56810687dc77 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:00.333564 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 884/1751] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:29:59 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 572 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:30:00.334492 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540070bb-1dee-4012-93f6-758ac5f73d6c service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:00.350583 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3368da60-cdec-4769-a62f-fa080a267c14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:00.351264 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3368da60-cdec-4769-a62f-fa080a267c14 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:00.351633 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3368da60-cdec-4769-a62f-fa080a267c14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:00.351907 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3368da60-cdec-4769-a62f-fa080a267c14 admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:00.352802 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3368da60-cdec-4769-a62f-fa080a267c14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:00.354759 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3368da60-cdec-4769-a62f-fa080a267c14 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:00.367342 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540070bb-1dee-4012-93f6-758ac5f73d6c service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:00.381781 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a586f8a-e294-45de-a4d1-03238d807eae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:00.382480 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a586f8a-e294-45de-a4d1-03238d807eae admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:00.382941 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a586f8a-e294-45de-a4d1-03238d807eae admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:00.383227 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a586f8a-e294-45de-a4d1-03238d807eae admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:00.384111 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a586f8a-e294-45de-a4d1-03238d807eae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:00.386088 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a586f8a-e294-45de-a4d1-03238d807eae admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:00.509548 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 868/1752] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 17 19:30:00 2022] POST /identity/v3/projects => generated 369 bytes in 411 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:30:00.514437 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3368da60-cdec-4769-a62f-fa080a267c14 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:00.542770 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 885/1753] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 17 19:30:00 2022] POST /identity/v3/users => generated 291 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:30:00.550366 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a586f8a-e294-45de-a4d1-03238d807eae admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:00.577524 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a586f8a-e294-45de-a4d1-03238d807eae admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:00.659961 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3368da60-cdec-4769-a62f-fa080a267c14 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:00.668320 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd5359b-3673-43d8-bd26-e0f085326927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:00.669397 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd5359b-3673-43d8-bd26-e0f085326927 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:00.669873 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd5359b-3673-43d8-bd26-e0f085326927 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:00.670865 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd5359b-3673-43d8-bd26-e0f085326927 admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:00.671821 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd5359b-3673-43d8-bd26-e0f085326927 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:00.674076 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd5359b-3673-43d8-bd26-e0f085326927 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:00.711919 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec6ef42-2755-4f94-829d-b73e900e0024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:00.714448 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec6ef42-2755-4f94-829d-b73e900e0024 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:00.714448 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec6ef42-2755-4f94-829d-b73e900e0024 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:00.714448 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec6ef42-2755-4f94-829d-b73e900e0024 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:00.714760 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec6ef42-2755-4f94-829d-b73e900e0024 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:00.716354 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec6ef42-2755-4f94-829d-b73e900e0024 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:00.737710 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 886/1754] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:30:00 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:30:00.743808 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec6ef42-2755-4f94-829d-b73e900e0024 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:00.778379 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec6ef42-2755-4f94-829d-b73e900e0024 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:00.789507 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e10d554-0c13-46e7-9769-c2cc5ac7d27b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:00.789507 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e10d554-0c13-46e7-9769-c2cc5ac7d27b admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:00.789507 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e10d554-0c13-46e7-9769-c2cc5ac7d27b admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:00.789507 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e10d554-0c13-46e7-9769-c2cc5ac7d27b admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:00.789888 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e10d554-0c13-46e7-9769-c2cc5ac7d27b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:00.791185 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e10d554-0c13-46e7-9769-c2cc5ac7d27b admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:00.804645 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 869/1755] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 17 19:30:00 2022] POST /identity/v3/users => generated 291 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 17 19:30:00.806626 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd5359b-3673-43d8-bd26-e0f085326927 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:00.811167 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 887/1756] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:30:00 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:30:00.812655 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e10d554-0c13-46e7-9769-c2cc5ac7d27b admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:00.826029 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e10d554-0c13-46e7-9769-c2cc5ac7d27b admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:00.835982 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1131415-5c9f-4bbb-a1e5-b461f2174fc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:00.837284 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1131415-5c9f-4bbb-a1e5-b461f2174fc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:00.837869 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1131415-5c9f-4bbb-a1e5-b461f2174fc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:00.838358 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1131415-5c9f-4bbb-a1e5-b461f2174fc8 admin admin] PATH_INFO: `/v3/projects/71337bc709d54773810da46d85f52d34/users/84877a41a7ee477dac6164f0efe7d65d/roles/114e6a16e63447d1924707ace5aace25` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:00.841045 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd5359b-3673-43d8-bd26-e0f085326927 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:00.867401 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5ea268-326e-4a5f-a869-5dc65004c5d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:00.868144 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5ea268-326e-4a5f-a869-5dc65004c5d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:00.868424 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5ea268-326e-4a5f-a869-5dc65004c5d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:00.868685 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5ea268-326e-4a5f-a869-5dc65004c5d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:00.869413 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5ea268-326e-4a5f-a869-5dc65004c5d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:00.871015 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5ea268-326e-4a5f-a869-5dc65004c5d4 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:00.883124 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1131415-5c9f-4bbb-a1e5-b461f2174fc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71337bc709d54773810da46d85f52d34, user_id=84877a41a7ee477dac6164f0efe7d65d, role_id=114e6a16e63447d1924707ace5aace25)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:00.886794 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1131415-5c9f-4bbb-a1e5-b461f2174fc8 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:00.914190 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 870/1757] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:30:00 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:30:00.915170 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5ea268-326e-4a5f-a869-5dc65004c5d4 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:00.940199 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5ea268-326e-4a5f-a869-5dc65004c5d4 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:00.949270 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57395b67-afb5-4049-8106-772dfc8cfbdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:00.950168 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57395b67-afb5-4049-8106-772dfc8cfbdd admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:00.950444 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57395b67-afb5-4049-8106-772dfc8cfbdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:00.956161 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57395b67-afb5-4049-8106-772dfc8cfbdd admin admin] PATH_INFO: `/v3/projects/55fa3d2caf52480c8ecee4c7fd511231/users/3aa3665c65434c7f8dc0c5a9a7590d96/roles/60af5088793e4379b88821a86552a344` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:00.990154 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57395b67-afb5-4049-8106-772dfc8cfbdd admin admin] RBAC: Authorizing `identity:create_grant(project_id=55fa3d2caf52480c8ecee4c7fd511231, user_id=3aa3665c65434c7f8dc0c5a9a7590d96, role_id=60af5088793e4379b88821a86552a344)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:00.992903 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57395b67-afb5-4049-8106-772dfc8cfbdd admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:01.030882 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 888/1758] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:30:00 2022] PUT /identity/v3/projects/71337bc709d54773810da46d85f52d34/users/84877a41a7ee477dac6164f0efe7d65d/roles/114e6a16e63447d1924707ace5aace25 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:30:01.042642 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ef2871-4153-4b0d-914f-7f3dba64529e None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:01.043740 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ef2871-4153-4b0d-914f-7f3dba64529e None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:01.044025 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ef2871-4153-4b0d-914f-7f3dba64529e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:01.099069 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 871/1759] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:30:00 2022] PUT /identity/v3/projects/55fa3d2caf52480c8ecee4c7fd511231/users/3aa3665c65434c7f8dc0c5a9a7590d96/roles/60af5088793e4379b88821a86552a344 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 17 19:30:01.100438 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-31ef2871-4153-4b0d-914f-7f3dba64529e None None] MFA Rules not processed for user `84877a41a7ee477dac6164f0efe7d65d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:30:01.102920 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57395b67-afb5-4049-8106-772dfc8cfbdd admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:01.103520 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-31ef2871-4153-4b0d-914f-7f3dba64529e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:01.246013 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57395b67-afb5-4049-8106-772dfc8cfbdd admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:01.261184 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a4c806-9c2a-4c71-8648-a9949cc5ec0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:01.263270 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a4c806-9c2a-4c71-8648-a9949cc5ec0d admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:01.263926 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a4c806-9c2a-4c71-8648-a9949cc5ec0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:01.264564 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a4c806-9c2a-4c71-8648-a9949cc5ec0d admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:01.265809 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a4c806-9c2a-4c71-8648-a9949cc5ec0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:01.268323 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a4c806-9c2a-4c71-8648-a9949cc5ec0d admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:01.269199 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 889/1760] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:30:01 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 17 19:30:01.276927 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ef2871-4153-4b0d-914f-7f3dba64529e None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:01.288942 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ef2871-4153-4b0d-914f-7f3dba64529e None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:01.299066 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 872/1761] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:30:01 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:30:01.305390 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a4c806-9c2a-4c71-8648-a9949cc5ec0d admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:01.309003 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3b0b64-184e-4452-ad82-b2e5c7cf35bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:01.318100 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3b0b64-184e-4452-ad82-b2e5c7cf35bf admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:01.318436 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3b0b64-184e-4452-ad82-b2e5c7cf35bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:01.318739 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3b0b64-184e-4452-ad82-b2e5c7cf35bf admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:01.320760 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3b0b64-184e-4452-ad82-b2e5c7cf35bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:01.325374 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3b0b64-184e-4452-ad82-b2e5c7cf35bf admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:01.342317 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a4c806-9c2a-4c71-8648-a9949cc5ec0d admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:01.351843 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2d3632-dfba-4ce2-9373-fffba87d2f5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:01.352756 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2d3632-dfba-4ce2-9373-fffba87d2f5e admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:01.355865 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2d3632-dfba-4ce2-9373-fffba87d2f5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:01.356386 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2d3632-dfba-4ce2-9373-fffba87d2f5e admin admin] PATH_INFO: `/v3/projects/55fa3d2caf52480c8ecee4c7fd511231/users/3aa3665c65434c7f8dc0c5a9a7590d96/roles/60af5088793e4379b88821a86552a344` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:01.360595 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2d3632-dfba-4ce2-9373-fffba87d2f5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=55fa3d2caf52480c8ecee4c7fd511231, user_id=3aa3665c65434c7f8dc0c5a9a7590d96, role_id=60af5088793e4379b88821a86552a344)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:01.365391 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2d3632-dfba-4ce2-9373-fffba87d2f5e admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:01.408009 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 873/1762] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:30:01 2022] PUT /identity/v3/projects/55fa3d2caf52480c8ecee4c7fd511231/users/3aa3665c65434c7f8dc0c5a9a7590d96/roles/60af5088793e4379b88821a86552a344 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 17 19:30:01.409607 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9238336e-1765-4cfe-8e13-ba0c67e82c1c None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:01.410180 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9238336e-1765-4cfe-8e13-ba0c67e82c1c None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:01.410534 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9238336e-1765-4cfe-8e13-ba0c67e82c1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:01.456569 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-9238336e-1765-4cfe-8e13-ba0c67e82c1c None None] MFA Rules not processed for user `3aa3665c65434c7f8dc0c5a9a7590d96`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:30:01.460914 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 890/1763] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 17 19:30:01 2022] POST /identity/v3/projects => generated 379 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 17 19:30:01.461078 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-9238336e-1765-4cfe-8e13-ba0c67e82c1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:01.462391 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3b0b64-184e-4452-ad82-b2e5c7cf35bf admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:01.605279 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 874/1764] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:30:01 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:30:01.656584 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9238336e-1765-4cfe-8e13-ba0c67e82c1c None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:01.669267 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3b0b64-184e-4452-ad82-b2e5c7cf35bf admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:01.695711 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6a0d2e-e29f-4c5f-8109-09eef430234c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:01.696404 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6a0d2e-e29f-4c5f-8109-09eef430234c admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:01.696733 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6a0d2e-e29f-4c5f-8109-09eef430234c admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:01.697009 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6a0d2e-e29f-4c5f-8109-09eef430234c admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:01.697882 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6a0d2e-e29f-4c5f-8109-09eef430234c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:01.699908 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6a0d2e-e29f-4c5f-8109-09eef430234c admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:01.827951 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 891/1765] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 17 19:30:01 2022] POST /identity/v3/users => generated 296 bytes in 366 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:30:01.828701 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6a0d2e-e29f-4c5f-8109-09eef430234c admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:01.833459 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-ea6a0d2e-e29f-4c5f-8109-09eef430234c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:01.876921 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9238336e-1765-4cfe-8e13-ba0c67e82c1c None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:01.901042 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6a0d2e-e29f-4c5f-8109-09eef430234c admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:01.901203 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33d82cb-c77c-4fef-81c6-838d87efec16 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd05409b38ae346458abfe41ebda1c59e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:01.902376 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33d82cb-c77c-4fef-81c6-838d87efec16 service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:01.905746 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33d82cb-c77c-4fef-81c6-838d87efec16 service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:01.906122 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33d82cb-c77c-4fef-81c6-838d87efec16 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:01.920640 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9065de6c-62c9-40b6-bdfe-a8f084874026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:01.921528 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9065de6c-62c9-40b6-bdfe-a8f084874026 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:01.921808 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9065de6c-62c9-40b6-bdfe-a8f084874026 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:01.922166 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9065de6c-62c9-40b6-bdfe-a8f084874026 admin admin] PATH_INFO: `/v3/projects/c3209fe3cf6b4daaba671bc658f8b1a6` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:01.924537 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9065de6c-62c9-40b6-bdfe-a8f084874026 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3209fe3cf6b4daaba671bc658f8b1a6)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:01.928599 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9065de6c-62c9-40b6-bdfe-a8f084874026 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:01.938142 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33d82cb-c77c-4fef-81c6-838d87efec16 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:01.944392 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33d82cb-c77c-4fef-81c6-838d87efec16 service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:01.989543 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 875/1766] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:30:01 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:30:01.990533 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33d82cb-c77c-4fef-81c6-838d87efec16 service cinder] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:02.024077 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33d82cb-c77c-4fef-81c6-838d87efec16 service cinder] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:02.046855 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ac5953-4090-40ac-b57e-7d78cf9d7f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:02.050177 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ac5953-4090-40ac-b57e-7d78cf9d7f6d admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:02.050923 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ac5953-4090-40ac-b57e-7d78cf9d7f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:02.051836 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ac5953-4090-40ac-b57e-7d78cf9d7f6d admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:02.061066 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ac5953-4090-40ac-b57e-7d78cf9d7f6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:02.068476 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ac5953-4090-40ac-b57e-7d78cf9d7f6d admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:02.105250 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 876/1767] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:30:01 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:30:02.108050 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ac5953-4090-40ac-b57e-7d78cf9d7f6d admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:02.136457 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ac5953-4090-40ac-b57e-7d78cf9d7f6d admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:02.196999 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-9065de6c-62c9-40b6-bdfe-a8f084874026 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3209fe3cf6b4daaba671bc658f8b1a6', 'request_id': 'req-9065de6c-62c9-40b6-bdfe-a8f084874026'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:02.237204 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-9065de6c-62c9-40b6-bdfe-a8f084874026 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3209fe3cf6b4daaba671bc658f8b1a6', 'request_id': 'req-9065de6c-62c9-40b6-bdfe-a8f084874026'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:02.321364 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620acb6f-8563-4b7b-83f3-4060afb95b80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:02.324161 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620acb6f-8563-4b7b-83f3-4060afb95b80 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:02.324683 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620acb6f-8563-4b7b-83f3-4060afb95b80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:02.325128 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620acb6f-8563-4b7b-83f3-4060afb95b80 admin admin] PATH_INFO: `/v3/projects/5cbd020abc4747c9aba7f8d0ddad5d02/users/6300d36694c84482afa1ef40c48b9259/roles/60af5088793e4379b88821a86552a344` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:02.365786 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620acb6f-8563-4b7b-83f3-4060afb95b80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cbd020abc4747c9aba7f8d0ddad5d02, user_id=6300d36694c84482afa1ef40c48b9259, role_id=60af5088793e4379b88821a86552a344)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:02.369817 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620acb6f-8563-4b7b-83f3-4060afb95b80 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:02.991985 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-9065de6c-62c9-40b6-bdfe-a8f084874026 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3209fe3cf6b4daaba671bc658f8b1a6', 'request_id': 'req-9065de6c-62c9-40b6-bdfe-a8f084874026'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:03.007811 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 877/1768] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:30:02 2022] PUT /identity/v3/projects/5cbd020abc4747c9aba7f8d0ddad5d02/users/6300d36694c84482afa1ef40c48b9259/roles/60af5088793e4379b88821a86552a344 => generated 0 bytes in 901 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 17 19:30:03.011327 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620acb6f-8563-4b7b-83f3-4060afb95b80 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:03.028551 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-9065de6c-62c9-40b6-bdfe-a8f084874026 admin admin] The token cache is being invalidate because project c3209fe3cf6b4daaba671bc658f8b1a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 17 19:30:03.029047 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-9065de6c-62c9-40b6-bdfe-a8f084874026 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:03.030626 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-9065de6c-62c9-40b6-bdfe-a8f084874026 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:03.038446 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 892/1769] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 17 19:30:01 2022] DELETE /identity/v3/projects/c3209fe3cf6b4daaba671bc658f8b1a6 => generated 0 bytes in 1207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:30:03.040668 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9065de6c-62c9-40b6-bdfe-a8f084874026 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:03.043262 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-9065de6c-62c9-40b6-bdfe-a8f084874026 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:03.113973 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9065de6c-62c9-40b6-bdfe-a8f084874026 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:03.141993 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620acb6f-8563-4b7b-83f3-4060afb95b80 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:03.147161 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f347f6b7-782e-42b6-b541-350cf962cce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:03.147939 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f347f6b7-782e-42b6-b541-350cf962cce4 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:03.148226 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f347f6b7-782e-42b6-b541-350cf962cce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:03.148511 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f347f6b7-782e-42b6-b541-350cf962cce4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:03.149227 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f347f6b7-782e-42b6-b541-350cf962cce4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:03.152059 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f347f6b7-782e-42b6-b541-350cf962cce4 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:03.173397 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 878/1770] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:30:03 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 17 19:30:03.173397 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f347f6b7-782e-42b6-b541-350cf962cce4 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:03.174758 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-f347f6b7-782e-42b6-b541-350cf962cce4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:03.218929 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3f5d1d-9fca-411c-ac38-ce80b7e449a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:03.219993 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3f5d1d-9fca-411c-ac38-ce80b7e449a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:03.220320 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3f5d1d-9fca-411c-ac38-ce80b7e449a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:03.220580 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3f5d1d-9fca-411c-ac38-ce80b7e449a2 admin admin] PATH_INFO: `/v3/users/a9d0d1805c2c47549990b42dc1f91500` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:03.222386 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3f5d1d-9fca-411c-ac38-ce80b7e449a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9d0d1805c2c47549990b42dc1f91500)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:03.225118 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3f5d1d-9fca-411c-ac38-ce80b7e449a2 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:03.329557 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f347f6b7-782e-42b6-b541-350cf962cce4 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:03.351606 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffa35d8-cd74-42be-bd30-c6bf4e5a9d61 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd05409b38ae346458abfe41ebda1c59e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:03.352441 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffa35d8-cd74-42be-bd30-c6bf4e5a9d61 service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:03.352798 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffa35d8-cd74-42be-bd30-c6bf4e5a9d61 service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:03.353143 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffa35d8-cd74-42be-bd30-c6bf4e5a9d61 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:03.360082 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-2ffa35d8-cd74-42be-bd30-c6bf4e5a9d61 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:03.491921 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-7f3f5d1d-9fca-411c-ac38-ce80b7e449a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9d0d1805c2c47549990b42dc1f91500'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:03.493138 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-7f3f5d1d-9fca-411c-ac38-ce80b7e449a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9d0d1805c2c47549990b42dc1f91500'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:03.531730 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-7f3f5d1d-9fca-411c-ac38-ce80b7e449a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9d0d1805c2c47549990b42dc1f91500'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:03.547204 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffa35d8-cd74-42be-bd30-c6bf4e5a9d61 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:03.552725 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffa35d8-cd74-42be-bd30-c6bf4e5a9d61 service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:03.578635 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 879/1771] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:30:03 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:30:03.580804 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffa35d8-cd74-42be-bd30-c6bf4e5a9d61 service cinder] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:03.584811 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-2ffa35d8-cd74-42be-bd30-c6bf4e5a9d61 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:03.619287 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffa35d8-cd74-42be-bd30-c6bf4e5a9d61 service cinder] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:03.632186 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcf8dca-e40d-450c-b926-bf2793efed69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:03.638302 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcf8dca-e40d-450c-b926-bf2793efed69 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:03.638302 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcf8dca-e40d-450c-b926-bf2793efed69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:03.638302 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcf8dca-e40d-450c-b926-bf2793efed69 admin admin] PATH_INFO: `/v3/projects/5cbd020abc4747c9aba7f8d0ddad5d02/users/6300d36694c84482afa1ef40c48b9259/roles/60af5088793e4379b88821a86552a344` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:03.638302 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcf8dca-e40d-450c-b926-bf2793efed69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cbd020abc4747c9aba7f8d0ddad5d02, user_id=6300d36694c84482afa1ef40c48b9259, role_id=60af5088793e4379b88821a86552a344)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:03.640332 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcf8dca-e40d-450c-b926-bf2793efed69 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:03.653295 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 880/1772] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:30:03 2022] PUT /identity/v3/projects/5cbd020abc4747c9aba7f8d0ddad5d02/users/6300d36694c84482afa1ef40c48b9259/roles/60af5088793e4379b88821a86552a344 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:30:03.655053 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bf40cd-6911-48f5-ae54-12869469c553 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:03.655436 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bf40cd-6911-48f5-ae54-12869469c553 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:03.655809 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bf40cd-6911-48f5-ae54-12869469c553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:03.667607 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-7f3f5d1d-9fca-411c-ac38-ce80b7e449a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9d0d1805c2c47549990b42dc1f91500'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:03.676908 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-7f3f5d1d-9fca-411c-ac38-ce80b7e449a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9d0d1805c2c47549990b42dc1f91500'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:03.685370 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 893/1773] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 17 19:30:03 2022] DELETE /identity/v3/users/a9d0d1805c2c47549990b42dc1f91500 => generated 0 bytes in 646 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:30:03.686961 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bafe46d-65d8-4440-afaf-7e84e77d9a71 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:03.687310 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bafe46d-65d8-4440-afaf-7e84e77d9a71 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:03.687610 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bafe46d-65d8-4440-afaf-7e84e77d9a71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:03.728610 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-d1bf40cd-6911-48f5-ae54-12869469c553 None None] MFA Rules not processed for user `6545bb5d5eb24c199b5d89091b2c1545`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:30:03.729833 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-d1bf40cd-6911-48f5-ae54-12869469c553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:03.734858 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-1bafe46d-65d8-4440-afaf-7e84e77d9a71 None None] MFA Rules not processed for user `6300d36694c84482afa1ef40c48b9259`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:30:03.736529 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-1bafe46d-65d8-4440-afaf-7e84e77d9a71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:03.839963 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 894/1774] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:30:03 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:30:03.920757 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 881/1775] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:30:03 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:30:04.780457 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bafe46d-65d8-4440-afaf-7e84e77d9a71 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:04.782403 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-1bafe46d-65d8-4440-afaf-7e84e77d9a71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:04.928928 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc11cf3-cb8f-4433-a208-7714e62baaed None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:04.929414 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc11cf3-cb8f-4433-a208-7714e62baaed None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:04.930249 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc11cf3-cb8f-4433-a208-7714e62baaed None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:04.931904 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 882/1776] 10.222.0.58 () {54 vars in 912 bytes} [Tue May 17 19:30:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:30:04.937488 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bafe46d-65d8-4440-afaf-7e84e77d9a71 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:04.940385 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556d287d-c896-47aa-bb36-34c3ddd15323 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:04.944961 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556d287d-c896-47aa-bb36-34c3ddd15323 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:04.944961 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556d287d-c896-47aa-bb36-34c3ddd15323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:04.948307 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592b9511-1bd0-4422-b526-d5e559db23bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:04.952930 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592b9511-1bd0-4422-b526-d5e559db23bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:04.953359 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592b9511-1bd0-4422-b526-d5e559db23bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:04.953647 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592b9511-1bd0-4422-b526-d5e559db23bd admin admin] PATH_INFO: `/v3/projects/2a3df6eacaef4fc3afc55a204cf6d07a` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:04.958757 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592b9511-1bd0-4422-b526-d5e559db23bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a3df6eacaef4fc3afc55a204cf6d07a)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:04.964391 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592b9511-1bd0-4422-b526-d5e559db23bd admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:04.967974 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-556d287d-c896-47aa-bb36-34c3ddd15323 None None] MFA Rules not processed for user `1ac193d15f48463c98ff1d97770ed794`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:30:04.970028 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-556d287d-c896-47aa-bb36-34c3ddd15323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:05.058172 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 883/1777] 10.222.0.58 () {58 vars in 1009 bytes} [Tue May 17 19:30:04 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 17 19:30:05.090155 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556d287d-c896-47aa-bb36-34c3ddd15323 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:05.091824 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-556d287d-c896-47aa-bb36-34c3ddd15323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:05.113675 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556d287d-c896-47aa-bb36-34c3ddd15323 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:05.135361 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-592b9511-1bd0-4422-b526-d5e559db23bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a3df6eacaef4fc3afc55a204cf6d07a', 'request_id': 'req-592b9511-1bd0-4422-b526-d5e559db23bd'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:05.145453 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-592b9511-1bd0-4422-b526-d5e559db23bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a3df6eacaef4fc3afc55a204cf6d07a', 'request_id': 'req-592b9511-1bd0-4422-b526-d5e559db23bd'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:05.220852 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1278cb84-b382-4617-aa7c-084c90524b6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:05.221818 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1278cb84-b382-4617-aa7c-084c90524b6d service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:05.222047 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1278cb84-b382-4617-aa7c-084c90524b6d service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:05.222474 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1278cb84-b382-4617-aa7c-084c90524b6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:05.238085 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1278cb84-b382-4617-aa7c-084c90524b6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:05.239372 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1278cb84-b382-4617-aa7c-084c90524b6d service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:05.241987 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-592b9511-1bd0-4422-b526-d5e559db23bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a3df6eacaef4fc3afc55a204cf6d07a', 'request_id': 'req-592b9511-1bd0-4422-b526-d5e559db23bd'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:05.262060 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-592b9511-1bd0-4422-b526-d5e559db23bd admin admin] The token cache is being invalidate because project 2a3df6eacaef4fc3afc55a204cf6d07a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 17 19:30:05.262349 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-592b9511-1bd0-4422-b526-d5e559db23bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:05.263065 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-592b9511-1bd0-4422-b526-d5e559db23bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:05.264158 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 884/1778] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:30:05 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 17 19:30:05.269914 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 895/1779] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 17 19:30:04 2022] DELETE /identity/v3/projects/2a3df6eacaef4fc3afc55a204cf6d07a => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:30:05.273036 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f631db-7778-4c1e-b9aa-3a147b52634f None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:05.273343 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f631db-7778-4c1e-b9aa-3a147b52634f None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:05.273623 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f631db-7778-4c1e-b9aa-3a147b52634f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:05.300147 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-93f631db-7778-4c1e-b9aa-3a147b52634f None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:30:05.308205 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-93f631db-7778-4c1e-b9aa-3a147b52634f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:05.486683 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 885/1780] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:30:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:30:05.487085 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e0e8a0-3a54-40b9-a41c-52ea27a06a62 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:05.487390 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e0e8a0-3a54-40b9-a41c-52ea27a06a62 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:05.487579 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e0e8a0-3a54-40b9-a41c-52ea27a06a62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:05.517013 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-80e0e8a0-3a54-40b9-a41c-52ea27a06a62 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:30:05.518504 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-80e0e8a0-3a54-40b9-a41c-52ea27a06a62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:05.529798 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 896/1781] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:30:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:30:05.535852 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f631db-7778-4c1e-b9aa-3a147b52634f None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:05.577845 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f631db-7778-4c1e-b9aa-3a147b52634f None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:05.590151 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdda3fab-514d-4d80-953d-2066eb6bf0bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:05.591866 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdda3fab-514d-4d80-953d-2066eb6bf0bd admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:05.594202 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdda3fab-514d-4d80-953d-2066eb6bf0bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:05.595133 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdda3fab-514d-4d80-953d-2066eb6bf0bd admin admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:05.596116 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdda3fab-514d-4d80-953d-2066eb6bf0bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:30:05.597293 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdda3fab-514d-4d80-953d-2066eb6bf0bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:05.601312 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdda3fab-514d-4d80-953d-2066eb6bf0bd admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:05.626103 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 886/1782] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 17 19:30:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:30:05.642675 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e0e8a0-3a54-40b9-a41c-52ea27a06a62 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:05.653830 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e0e8a0-3a54-40b9-a41c-52ea27a06a62 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:05.677423 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd3dbee-464b-43d3-ad5a-87fb41efc650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:05.679752 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd3dbee-464b-43d3-ad5a-87fb41efc650 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:05.680109 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd3dbee-464b-43d3-ad5a-87fb41efc650 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:05.682382 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd3dbee-464b-43d3-ad5a-87fb41efc650 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:05.683220 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd3dbee-464b-43d3-ad5a-87fb41efc650 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:05.707788 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd3dbee-464b-43d3-ad5a-87fb41efc650 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:05.892550 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 897/1783] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 17 19:30:05 2022] POST /identity/v3/projects => generated 381 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:30:05.894821 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdda3fab-514d-4d80-953d-2066eb6bf0bd admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:05.909903 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdda3fab-514d-4d80-953d-2066eb6bf0bd admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:06.040297 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1c618b-3b61-4cf3-a878-0c78d3688a53 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:06.040598 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1c618b-3b61-4cf3-a878-0c78d3688a53 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:06.040876 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1c618b-3b61-4cf3-a878-0c78d3688a53 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:06.042251 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 898/1784] 10.222.0.58 () {54 vars in 921 bytes} [Tue May 17 19:30:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:30:06.081635 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1c618b-3b61-4cf3-a878-0c78d3688a53 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:06.084241 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-fd1c618b-3b61-4cf3-a878-0c78d3688a53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:06.106263 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965ef5a8-7dc9-48dc-baaf-b10d7e56c5f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:06.108985 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965ef5a8-7dc9-48dc-baaf-b10d7e56c5f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:06.110729 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965ef5a8-7dc9-48dc-baaf-b10d7e56c5f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:06.110729 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965ef5a8-7dc9-48dc-baaf-b10d7e56c5f2 admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:06.112262 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965ef5a8-7dc9-48dc-baaf-b10d7e56c5f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:06.114842 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965ef5a8-7dc9-48dc-baaf-b10d7e56c5f2 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:06.272188 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 887/1785] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 17 19:30:05 2022] POST /identity/v3/users => generated 291 bytes in 379 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:30:06.275578 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6baeb7e-cc23-4968-9c79-7ce501f23e87 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:06.288405 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6baeb7e-cc23-4968-9c79-7ce501f23e87 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:06.288405 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6baeb7e-cc23-4968-9c79-7ce501f23e87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:06.289026 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1c618b-3b61-4cf3-a878-0c78d3688a53 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:06.290524 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98da1bd-b631-496f-b4d3-7590e006b23d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:06.291519 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98da1bd-b631-496f-b4d3-7590e006b23d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:06.291839 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98da1bd-b631-496f-b4d3-7590e006b23d admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:06.292177 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98da1bd-b631-496f-b4d3-7590e006b23d admin admin] PATH_INFO: `/v3/users/8b8bfa2f51cb4333897752cce6b5026b` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:06.295082 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98da1bd-b631-496f-b4d3-7590e006b23d admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b8bfa2f51cb4333897752cce6b5026b)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:06.297255 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98da1bd-b631-496f-b4d3-7590e006b23d admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:06.366900 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-c6baeb7e-cc23-4968-9c79-7ce501f23e87 None None] MFA Rules not processed for user `65c1f878b0dc4d63a4f5dcb12fac1dc1`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:30:06.375884 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-c6baeb7e-cc23-4968-9c79-7ce501f23e87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:06.571842 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 888/1786] 10.222.0.58 () {58 vars in 1018 bytes} [Tue May 17 19:30:06 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:30:06.573474 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6baeb7e-cc23-4968-9c79-7ce501f23e87 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:06.610213 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6baeb7e-cc23-4968-9c79-7ce501f23e87 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:06.621490 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23645329-3beb-4b0c-8e4c-847a8530bc2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:06.622310 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23645329-3beb-4b0c-8e4c-847a8530bc2c admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:06.622794 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23645329-3beb-4b0c-8e4c-847a8530bc2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:06.623184 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23645329-3beb-4b0c-8e4c-847a8530bc2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:06.624110 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23645329-3beb-4b0c-8e4c-847a8530bc2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:06.629001 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23645329-3beb-4b0c-8e4c-847a8530bc2c admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:06.659254 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 889/1787] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:30:06 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:30:06.664493 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23645329-3beb-4b0c-8e4c-847a8530bc2c admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:06.685888 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23645329-3beb-4b0c-8e4c-847a8530bc2c admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:06.691445 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4723fc8b-4418-490a-ac6e-268f8e90deb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:06.692311 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4723fc8b-4418-490a-ac6e-268f8e90deb1 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:06.692593 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4723fc8b-4418-490a-ac6e-268f8e90deb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:06.692867 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4723fc8b-4418-490a-ac6e-268f8e90deb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:06.693611 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4723fc8b-4418-490a-ac6e-268f8e90deb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:06.695234 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4723fc8b-4418-490a-ac6e-268f8e90deb1 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:06.709800 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 890/1788] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:30:06 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:30:06.720185 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4723fc8b-4418-490a-ac6e-268f8e90deb1 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:06.731255 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4723fc8b-4418-490a-ac6e-268f8e90deb1 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:06.777369 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3029fa78-2cf3-4cbd-abc9-9598f212ca32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:06.777369 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3029fa78-2cf3-4cbd-abc9-9598f212ca32 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:06.777369 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3029fa78-2cf3-4cbd-abc9-9598f212ca32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:06.777369 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3029fa78-2cf3-4cbd-abc9-9598f212ca32 admin admin] PATH_INFO: `/v3/projects/32020b38b382462d8e702c42972d2d1d/users/d78743112b7b4e6f90dab0daa6268664/roles/114e6a16e63447d1924707ace5aace25` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:06.778067 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3029fa78-2cf3-4cbd-abc9-9598f212ca32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=32020b38b382462d8e702c42972d2d1d, user_id=d78743112b7b4e6f90dab0daa6268664, role_id=114e6a16e63447d1924707ace5aace25)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:06.778067 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3029fa78-2cf3-4cbd-abc9-9598f212ca32 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:06.995880 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 891/1789] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:30:06 2022] PUT /identity/v3/projects/32020b38b382462d8e702c42972d2d1d/users/d78743112b7b4e6f90dab0daa6268664/roles/114e6a16e63447d1924707ace5aace25 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:30:07.000285 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3029fa78-2cf3-4cbd-abc9-9598f212ca32 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:07.004338 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-3029fa78-2cf3-4cbd-abc9-9598f212ca32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:07.115360 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c98da1bd-b631-496f-b4d3-7590e006b23d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b8bfa2f51cb4333897752cce6b5026b'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:07.131764 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c98da1bd-b631-496f-b4d3-7590e006b23d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b8bfa2f51cb4333897752cce6b5026b'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:07.200457 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c98da1bd-b631-496f-b4d3-7590e006b23d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b8bfa2f51cb4333897752cce6b5026b'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:07.230065 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3029fa78-2cf3-4cbd-abc9-9598f212ca32 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:07.254903 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f103625-a0e6-4bbd-a612-4c140f3f5f5d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c1f878b0dc4d63a4f5dcb12fac1dc1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:07.256476 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f103625-a0e6-4bbd-a612-4c140f3f5f5d service nova] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:07.256884 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f103625-a0e6-4bbd-a612-4c140f3f5f5d service nova] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:07.258814 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f103625-a0e6-4bbd-a612-4c140f3f5f5d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:07.264064 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-4f103625-a0e6-4bbd-a612-4c140f3f5f5d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:07.319993 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f103625-a0e6-4bbd-a612-4c140f3f5f5d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:07.321708 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f103625-a0e6-4bbd-a612-4c140f3f5f5d service nova] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:07.327550 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c98da1bd-b631-496f-b4d3-7590e006b23d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b8bfa2f51cb4333897752cce6b5026b'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:07.336396 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c98da1bd-b631-496f-b4d3-7590e006b23d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b8bfa2f51cb4333897752cce6b5026b'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:07.351417 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 899/1790] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 17 19:30:06 2022] DELETE /identity/v3/users/8b8bfa2f51cb4333897752cce6b5026b => generated 0 bytes in 1270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:30:07.358271 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa9fa09-53c9-4f2d-be20-6174c102c17b None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:07.358448 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa9fa09-53c9-4f2d-be20-6174c102c17b None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:07.358866 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa9fa09-53c9-4f2d-be20-6174c102c17b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:07.377486 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 892/1791] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:30:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:30:07.378922 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f81bd7-3267-4aa5-908f-3384ffad41ee None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:07.383425 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f81bd7-3267-4aa5-908f-3384ffad41ee None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:07.387176 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f81bd7-3267-4aa5-908f-3384ffad41ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:07.464443 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-7aa9fa09-53c9-4f2d-be20-6174c102c17b None None] MFA Rules not processed for user `d78743112b7b4e6f90dab0daa6268664`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:30:07.464855 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-a7f81bd7-3267-4aa5-908f-3384ffad41ee None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:30:07.466237 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-7aa9fa09-53c9-4f2d-be20-6174c102c17b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:07.470276 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-a7f81bd7-3267-4aa5-908f-3384ffad41ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:07.606982 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 900/1792] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:30:07 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:30:07.612743 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa9fa09-53c9-4f2d-be20-6174c102c17b None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:07.616030 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-7aa9fa09-53c9-4f2d-be20-6174c102c17b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:07.698373 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 893/1793] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:30:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:30:07.730335 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f81bd7-3267-4aa5-908f-3384ffad41ee None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:07.737276 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-a7f81bd7-3267-4aa5-908f-3384ffad41ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:07.803583 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa9fa09-53c9-4f2d-be20-6174c102c17b None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:07.815796 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a5568b-d680-4096-a19b-658b273eef47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:07.816864 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a5568b-d680-4096-a19b-658b273eef47 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:07.818571 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a5568b-d680-4096-a19b-658b273eef47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:07.819081 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a5568b-d680-4096-a19b-658b273eef47 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:07.820136 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a5568b-d680-4096-a19b-658b273eef47 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:07.825340 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a5568b-d680-4096-a19b-658b273eef47 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:07.854760 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f81bd7-3267-4aa5-908f-3384ffad41ee None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:07.866178 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca574a30-9da6-4af0-93a5-09d6ef27678a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:07.867246 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca574a30-9da6-4af0-93a5-09d6ef27678a service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:07.867829 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca574a30-9da6-4af0-93a5-09d6ef27678a service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:07.868408 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca574a30-9da6-4af0-93a5-09d6ef27678a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:07.882657 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca574a30-9da6-4af0-93a5-09d6ef27678a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:07.885052 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca574a30-9da6-4af0-93a5-09d6ef27678a service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:07.901624 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 894/1794] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:30:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:30:07.942043 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 901/1795] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 17 19:30:07 2022] POST /identity/v3/projects => generated 381 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:30:07.951114 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca574a30-9da6-4af0-93a5-09d6ef27678a service neutron] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:07.985424 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca574a30-9da6-4af0-93a5-09d6ef27678a service neutron] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:08.071286 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8014df-ccae-4859-9b66-e2aa0388b170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:08.072854 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8014df-ccae-4859-9b66-e2aa0388b170 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:08.072854 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8014df-ccae-4859-9b66-e2aa0388b170 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:08.073437 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8014df-ccae-4859-9b66-e2aa0388b170 admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:08.075343 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8014df-ccae-4859-9b66-e2aa0388b170 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:08.078146 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8014df-ccae-4859-9b66-e2aa0388b170 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:08.153449 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a5568b-d680-4096-a19b-658b273eef47 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:08.155521 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-e0a5568b-d680-4096-a19b-658b273eef47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:08.171289 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a5568b-d680-4096-a19b-658b273eef47 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:08.180207 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab836528-966b-44ed-b917-6916eaa8987e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:08.181523 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab836528-966b-44ed-b917-6916eaa8987e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:08.181874 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab836528-966b-44ed-b917-6916eaa8987e admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:08.183222 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab836528-966b-44ed-b917-6916eaa8987e admin admin] PATH_INFO: `/v3/users/b04c599f7a054d4b91aba25701a3ed5f` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:08.183615 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab836528-966b-44ed-b917-6916eaa8987e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b04c599f7a054d4b91aba25701a3ed5f)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:08.186413 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab836528-966b-44ed-b917-6916eaa8987e admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:08.264414 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 895/1796] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 17 19:30:07 2022] POST /identity/v3/users => generated 297 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:30:08.268456 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8014df-ccae-4859-9b66-e2aa0388b170 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:08.283858 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8014df-ccae-4859-9b66-e2aa0388b170 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:08.289966 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6050678e-b7ab-4da4-b2eb-59de8c50032a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:08.290949 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6050678e-b7ab-4da4-b2eb-59de8c50032a admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:08.291348 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6050678e-b7ab-4da4-b2eb-59de8c50032a admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:08.291876 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6050678e-b7ab-4da4-b2eb-59de8c50032a admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:08.292808 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6050678e-b7ab-4da4-b2eb-59de8c50032a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:08.295961 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6050678e-b7ab-4da4-b2eb-59de8c50032a admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:08.333596 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 896/1797] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:30:08 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:30:08.338498 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6050678e-b7ab-4da4-b2eb-59de8c50032a admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:08.350478 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6050678e-b7ab-4da4-b2eb-59de8c50032a admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:08.365494 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50c61b8-dde6-4083-b080-b3d743bd8b78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:08.385136 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50c61b8-dde6-4083-b080-b3d743bd8b78 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:08.385292 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50c61b8-dde6-4083-b080-b3d743bd8b78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:08.385550 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50c61b8-dde6-4083-b080-b3d743bd8b78 admin admin] PATH_INFO: `/v3/projects/20aaad0091b140d688414a4ae8cb61d8/users/6cdfe21105c74d4299e7fe2f9b4ac23d/roles/60af5088793e4379b88821a86552a344` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:08.405613 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50c61b8-dde6-4083-b080-b3d743bd8b78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20aaad0091b140d688414a4ae8cb61d8, user_id=6cdfe21105c74d4299e7fe2f9b4ac23d, role_id=60af5088793e4379b88821a86552a344)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:08.409450 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50c61b8-dde6-4083-b080-b3d743bd8b78 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:08.442902 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ab836528-966b-44ed-b917-6916eaa8987e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b04c599f7a054d4b91aba25701a3ed5f'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:08.443600 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ab836528-966b-44ed-b917-6916eaa8987e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b04c599f7a054d4b91aba25701a3ed5f'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:08.466969 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ab836528-966b-44ed-b917-6916eaa8987e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b04c599f7a054d4b91aba25701a3ed5f'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:08.493385 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 897/1798] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:30:08 2022] PUT /identity/v3/projects/20aaad0091b140d688414a4ae8cb61d8/users/6cdfe21105c74d4299e7fe2f9b4ac23d/roles/60af5088793e4379b88821a86552a344 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:30:08.497304 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50c61b8-dde6-4083-b080-b3d743bd8b78 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:08.499567 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-c50c61b8-dde6-4083-b080-b3d743bd8b78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:08.542524 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ab836528-966b-44ed-b917-6916eaa8987e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b04c599f7a054d4b91aba25701a3ed5f'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:08.550461 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ab836528-966b-44ed-b917-6916eaa8987e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b04c599f7a054d4b91aba25701a3ed5f'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:08.556624 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 902/1799] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 17 19:30:08 2022] DELETE /identity/v3/users/b04c599f7a054d4b91aba25701a3ed5f => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 17 19:30:08.561030 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a7cb8f-949a-40ac-a245-9811217e4d31 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:08.561385 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a7cb8f-949a-40ac-a245-9811217e4d31 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:08.561726 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a7cb8f-949a-40ac-a245-9811217e4d31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:08.565932 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50c61b8-dde6-4083-b080-b3d743bd8b78 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:08.579439 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-60a7cb8f-949a-40ac-a245-9811217e4d31 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:30:08.580924 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-60a7cb8f-949a-40ac-a245-9811217e4d31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:08.648660 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 903/1800] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:30:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 17 19:30:08.649071 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279ab0ea-926d-4d68-a334-0e6bc96bc13b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:08.649673 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a7cb8f-949a-40ac-a245-9811217e4d31 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:08.649806 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279ab0ea-926d-4d68-a334-0e6bc96bc13b admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:08.650142 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279ab0ea-926d-4d68-a334-0e6bc96bc13b admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:08.650480 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279ab0ea-926d-4d68-a334-0e6bc96bc13b admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:08.651244 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279ab0ea-926d-4d68-a334-0e6bc96bc13b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:08.651477 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-60a7cb8f-949a-40ac-a245-9811217e4d31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:08.653078 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279ab0ea-926d-4d68-a334-0e6bc96bc13b admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:08.664143 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a7cb8f-949a-40ac-a245-9811217e4d31 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:08.664838 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 898/1801] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:30:08 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 17 19:30:08.665787 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279ab0ea-926d-4d68-a334-0e6bc96bc13b admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:08.667392 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-279ab0ea-926d-4d68-a334-0e6bc96bc13b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:08.674151 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559ff5ea-518b-4775-850e-d12d752559c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:08.674151 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559ff5ea-518b-4775-850e-d12d752559c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:08.674151 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559ff5ea-518b-4775-850e-d12d752559c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:08.674151 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559ff5ea-518b-4775-850e-d12d752559c4 admin admin] PATH_INFO: `/v3/projects/95b5d609c5964ab6a459ce08fea64ce4` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:08.674487 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559ff5ea-518b-4775-850e-d12d752559c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=95b5d609c5964ab6a459ce08fea64ce4)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:08.674487 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559ff5ea-518b-4775-850e-d12d752559c4 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:08.725881 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279ab0ea-926d-4d68-a334-0e6bc96bc13b admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:08.729607 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f614be-8ba2-4fd5-9ae0-620414ca1829 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:08.731613 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f614be-8ba2-4fd5-9ae0-620414ca1829 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:08.731613 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f614be-8ba2-4fd5-9ae0-620414ca1829 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:08.731613 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f614be-8ba2-4fd5-9ae0-620414ca1829 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:08.740164 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f614be-8ba2-4fd5-9ae0-620414ca1829 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:08.741576 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f614be-8ba2-4fd5-9ae0-620414ca1829 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:08.750944 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 899/1802] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:30:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:30:08.751846 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f614be-8ba2-4fd5-9ae0-620414ca1829 service neutron] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:08.762218 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f614be-8ba2-4fd5-9ae0-620414ca1829 service neutron] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:08.767796 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b10575-452d-408d-90fc-4b34b1d43317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:08.768776 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b10575-452d-408d-90fc-4b34b1d43317 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:08.769049 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b10575-452d-408d-90fc-4b34b1d43317 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:08.769320 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b10575-452d-408d-90fc-4b34b1d43317 admin admin] PATH_INFO: `/v3/projects/20aaad0091b140d688414a4ae8cb61d8/users/6cdfe21105c74d4299e7fe2f9b4ac23d/roles/60af5088793e4379b88821a86552a344` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:08.776539 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b10575-452d-408d-90fc-4b34b1d43317 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20aaad0091b140d688414a4ae8cb61d8, user_id=6cdfe21105c74d4299e7fe2f9b4ac23d, role_id=60af5088793e4379b88821a86552a344)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:08.779953 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b10575-452d-408d-90fc-4b34b1d43317 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:08.796147 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 900/1803] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:30:08 2022] PUT /identity/v3/projects/20aaad0091b140d688414a4ae8cb61d8/users/6cdfe21105c74d4299e7fe2f9b4ac23d/roles/60af5088793e4379b88821a86552a344 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:30:08.804627 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1115fab2-9893-4450-ac49-cce76d4cc0b4 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:08.805583 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1115fab2-9893-4450-ac49-cce76d4cc0b4 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:08.805866 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1115fab2-9893-4450-ac49-cce76d4cc0b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:08.806479 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-559ff5ea-518b-4775-850e-d12d752559c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95b5d609c5964ab6a459ce08fea64ce4', 'request_id': 'req-559ff5ea-518b-4775-850e-d12d752559c4'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:08.819282 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-559ff5ea-518b-4775-850e-d12d752559c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95b5d609c5964ab6a459ce08fea64ce4', 'request_id': 'req-559ff5ea-518b-4775-850e-d12d752559c4'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:08.838041 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-1115fab2-9893-4450-ac49-cce76d4cc0b4 None None] MFA Rules not processed for user `6cdfe21105c74d4299e7fe2f9b4ac23d`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:30:08.839225 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-1115fab2-9893-4450-ac49-cce76d4cc0b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:08.910299 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 901/1804] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:30:08 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:30:08.912330 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-559ff5ea-518b-4775-850e-d12d752559c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95b5d609c5964ab6a459ce08fea64ce4', 'request_id': 'req-559ff5ea-518b-4775-850e-d12d752559c4'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:08.922038 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1115fab2-9893-4450-ac49-cce76d4cc0b4 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:08.923480 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-1115fab2-9893-4450-ac49-cce76d4cc0b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:08.925404 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-559ff5ea-518b-4775-850e-d12d752559c4 admin admin] The token cache is being invalidate because project 95b5d609c5964ab6a459ce08fea64ce4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 17 19:30:08.925700 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-559ff5ea-518b-4775-850e-d12d752559c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:08.926356 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-559ff5ea-518b-4775-850e-d12d752559c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:08.928093 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 904/1805] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 17 19:30:08 2022] DELETE /identity/v3/projects/95b5d609c5964ab6a459ce08fea64ce4 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:30:08.932808 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3665496a-d6b7-43fb-a62b-7a9e9e0829ff None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:08.933088 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3665496a-d6b7-43fb-a62b-7a9e9e0829ff None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:08.933532 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3665496a-d6b7-43fb-a62b-7a9e9e0829ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:08.957744 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-3665496a-d6b7-43fb-a62b-7a9e9e0829ff None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:30:08.959117 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-3665496a-d6b7-43fb-a62b-7a9e9e0829ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:08.999208 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1115fab2-9893-4450-ac49-cce76d4cc0b4 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:09.004936 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bac095-b401-4e24-a7d8-401d2fb9ff92 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd05409b38ae346458abfe41ebda1c59e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:09.006307 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bac095-b401-4e24-a7d8-401d2fb9ff92 service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:09.006806 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bac095-b401-4e24-a7d8-401d2fb9ff92 service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:09.007087 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bac095-b401-4e24-a7d8-401d2fb9ff92 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:09.010969 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-22bac095-b401-4e24-a7d8-401d2fb9ff92 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:09.024598 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 905/1806] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:30:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 17 19:30:09.031812 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bac095-b401-4e24-a7d8-401d2fb9ff92 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:09.031911 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859e383f-a491-462f-9cb6-42ea306e588d None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:09.031911 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859e383f-a491-462f-9cb6-42ea306e588d None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:09.031911 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859e383f-a491-462f-9cb6-42ea306e588d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:09.032146 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bac095-b401-4e24-a7d8-401d2fb9ff92 service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:09.048106 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 902/1807] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:30:08 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 17 19:30:09.054140 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-859e383f-a491-462f-9cb6-42ea306e588d None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:30:09.055817 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-859e383f-a491-462f-9cb6-42ea306e588d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:09.067270 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 906/1808] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:30:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:30:09.071921 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bac095-b401-4e24-a7d8-401d2fb9ff92 service cinder] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:09.083821 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bac095-b401-4e24-a7d8-401d2fb9ff92 service cinder] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:09.089868 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a24ec7-a265-44c9-b339-34f3b5225390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:09.090620 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a24ec7-a265-44c9-b339-34f3b5225390 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:09.090963 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a24ec7-a265-44c9-b339-34f3b5225390 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:09.091299 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a24ec7-a265-44c9-b339-34f3b5225390 admin admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:09.092720 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a24ec7-a265-44c9-b339-34f3b5225390 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:30:09.093075 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a24ec7-a265-44c9-b339-34f3b5225390 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:09.094871 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a24ec7-a265-44c9-b339-34f3b5225390 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:09.110433 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 903/1809] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 17 19:30:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:30:09.114952 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859e383f-a491-462f-9cb6-42ea306e588d None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:09.135897 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859e383f-a491-462f-9cb6-42ea306e588d None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:09.138691 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c47cbe-e80a-4780-bbc0-c59b5925e84d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:09.139599 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c47cbe-e80a-4780-bbc0-c59b5925e84d admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:09.139994 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c47cbe-e80a-4780-bbc0-c59b5925e84d admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:09.140329 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c47cbe-e80a-4780-bbc0-c59b5925e84d admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:09.141117 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c47cbe-e80a-4780-bbc0-c59b5925e84d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:09.143216 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c47cbe-e80a-4780-bbc0-c59b5925e84d admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:09.172671 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a24ec7-a265-44c9-b339-34f3b5225390 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:09.174501 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-36a24ec7-a265-44c9-b339-34f3b5225390 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:09.189300 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a24ec7-a265-44c9-b339-34f3b5225390 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:09.193282 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80938530-638a-40f5-bc84-d921aa84c215 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd05409b38ae346458abfe41ebda1c59e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:09.194109 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80938530-638a-40f5-bc84-d921aa84c215 service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:09.194400 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80938530-638a-40f5-bc84-d921aa84c215 service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:09.194685 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80938530-638a-40f5-bc84-d921aa84c215 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:09.196597 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-80938530-638a-40f5-bc84-d921aa84c215 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:09.244154 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 907/1810] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 17 19:30:09 2022] POST /identity/v3/projects => generated 375 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:30:09.248703 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c47cbe-e80a-4780-bbc0-c59b5925e84d admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:09.269520 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80938530-638a-40f5-bc84-d921aa84c215 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:09.270883 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80938530-638a-40f5-bc84-d921aa84c215 service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:09.281810 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 904/1811] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:30:09 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 17 19:30:09.300440 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80938530-638a-40f5-bc84-d921aa84c215 service cinder] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:09.302703 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-80938530-638a-40f5-bc84-d921aa84c215 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:09.317435 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c47cbe-e80a-4780-bbc0-c59b5925e84d admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:09.324163 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936ac9f9-5318-48f7-a5e5-bb0247a2b927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:09.325379 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936ac9f9-5318-48f7-a5e5-bb0247a2b927 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:09.328350 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936ac9f9-5318-48f7-a5e5-bb0247a2b927 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:09.328350 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936ac9f9-5318-48f7-a5e5-bb0247a2b927 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:09.328350 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936ac9f9-5318-48f7-a5e5-bb0247a2b927 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:09.329771 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936ac9f9-5318-48f7-a5e5-bb0247a2b927 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:09.330719 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80938530-638a-40f5-bc84-d921aa84c215 service cinder] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:09.349189 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08c6ab5-31ae-4f05-a935-eeebce1221d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:09.350167 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08c6ab5-31ae-4f05-a935-eeebce1221d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:09.350449 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08c6ab5-31ae-4f05-a935-eeebce1221d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:09.350718 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08c6ab5-31ae-4f05-a935-eeebce1221d8 admin admin] PATH_INFO: `/v3/projects/1321d17bd52e406bb671194b9fab5173` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:09.351938 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08c6ab5-31ae-4f05-a935-eeebce1221d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1321d17bd52e406bb671194b9fab5173)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:09.353826 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08c6ab5-31ae-4f05-a935-eeebce1221d8 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:09.427647 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 908/1812] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 17 19:30:09 2022] POST /identity/v3/users => generated 288 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:30:09.431877 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936ac9f9-5318-48f7-a5e5-bb0247a2b927 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:09.441935 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936ac9f9-5318-48f7-a5e5-bb0247a2b927 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:09.447870 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb6e14b-d70b-483d-97ca-6bba3fe09a83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:09.448667 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb6e14b-d70b-483d-97ca-6bba3fe09a83 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:09.449069 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb6e14b-d70b-483d-97ca-6bba3fe09a83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:09.449419 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb6e14b-d70b-483d-97ca-6bba3fe09a83 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:09.450263 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb6e14b-d70b-483d-97ca-6bba3fe09a83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:09.452415 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb6e14b-d70b-483d-97ca-6bba3fe09a83 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:09.463245 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 909/1813] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:30:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 17 19:30:09.467661 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb6e14b-d70b-483d-97ca-6bba3fe09a83 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:09.479336 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb6e14b-d70b-483d-97ca-6bba3fe09a83 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:09.494757 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-b08c6ab5-31ae-4f05-a935-eeebce1221d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1321d17bd52e406bb671194b9fab5173', 'request_id': 'req-b08c6ab5-31ae-4f05-a935-eeebce1221d8'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:09.497422 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-b08c6ab5-31ae-4f05-a935-eeebce1221d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1321d17bd52e406bb671194b9fab5173', 'request_id': 'req-b08c6ab5-31ae-4f05-a935-eeebce1221d8'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:09.511031 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-b08c6ab5-31ae-4f05-a935-eeebce1221d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1321d17bd52e406bb671194b9fab5173', 'request_id': 'req-b08c6ab5-31ae-4f05-a935-eeebce1221d8'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:09.552851 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea63876-bc6b-4779-8f8b-bf8f6c37d2e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:09.553458 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea63876-bc6b-4779-8f8b-bf8f6c37d2e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:09.553738 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea63876-bc6b-4779-8f8b-bf8f6c37d2e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:09.554010 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea63876-bc6b-4779-8f8b-bf8f6c37d2e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:09.554745 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea63876-bc6b-4779-8f8b-bf8f6c37d2e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:09.556628 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea63876-bc6b-4779-8f8b-bf8f6c37d2e2 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:09.592206 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 910/1814] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:30:09 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 17 19:30:09.594062 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea63876-bc6b-4779-8f8b-bf8f6c37d2e2 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:09.602397 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-b08c6ab5-31ae-4f05-a935-eeebce1221d8 admin admin] The token cache is being invalidate because project 1321d17bd52e406bb671194b9fab5173 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 17 19:30:09.602838 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-b08c6ab5-31ae-4f05-a935-eeebce1221d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:09.603587 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-b08c6ab5-31ae-4f05-a935-eeebce1221d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:09.606280 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 905/1815] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 17 19:30:09 2022] DELETE /identity/v3/projects/1321d17bd52e406bb671194b9fab5173 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 17 19:30:09.608954 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea63876-bc6b-4779-8f8b-bf8f6c37d2e2 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:09.610769 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08c6ab5-31ae-4f05-a935-eeebce1221d8 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:09.612355 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-b08c6ab5-31ae-4f05-a935-eeebce1221d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:09.618452 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db13f187-b388-40e3-895d-5943761a7c5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:09.619386 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db13f187-b388-40e3-895d-5943761a7c5e admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:09.619701 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db13f187-b388-40e3-895d-5943761a7c5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:09.620112 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db13f187-b388-40e3-895d-5943761a7c5e admin admin] PATH_INFO: `/v3/projects/d758604b15e942098679fcdf859a699c/users/051e3b0893334b8e8f6064bcb266ea64/roles/114e6a16e63447d1924707ace5aace25` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:09.629528 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08c6ab5-31ae-4f05-a935-eeebce1221d8 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:09.642672 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d42b5f3-35a1-48ef-a0c1-5e2cbd3033b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:09.642776 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db13f187-b388-40e3-895d-5943761a7c5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d758604b15e942098679fcdf859a699c, user_id=051e3b0893334b8e8f6064bcb266ea64, role_id=114e6a16e63447d1924707ace5aace25)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:09.642857 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d42b5f3-35a1-48ef-a0c1-5e2cbd3033b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:09.642857 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d42b5f3-35a1-48ef-a0c1-5e2cbd3033b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:09.642857 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d42b5f3-35a1-48ef-a0c1-5e2cbd3033b1 admin admin] PATH_INFO: `/v3/users/3aa3665c65434c7f8dc0c5a9a7590d96` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:09.642857 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d42b5f3-35a1-48ef-a0c1-5e2cbd3033b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3aa3665c65434c7f8dc0c5a9a7590d96)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:09.643322 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db13f187-b388-40e3-895d-5943761a7c5e admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:09.645064 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d42b5f3-35a1-48ef-a0c1-5e2cbd3033b1 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:09.724928 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 911/1816] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:30:09 2022] PUT /identity/v3/projects/d758604b15e942098679fcdf859a699c/users/051e3b0893334b8e8f6064bcb266ea64/roles/114e6a16e63447d1924707ace5aace25 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:30:09.736263 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd167324-5aba-440d-aadd-9bf5344de22e None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:09.736665 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd167324-5aba-440d-aadd-9bf5344de22e None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:09.737013 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd167324-5aba-440d-aadd-9bf5344de22e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:09.767122 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-bd167324-5aba-440d-aadd-9bf5344de22e None None] MFA Rules not processed for user `051e3b0893334b8e8f6064bcb266ea64`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:30:09.768404 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-bd167324-5aba-440d-aadd-9bf5344de22e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:09.839117 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 912/1817] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:30:09 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:30:09.844878 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd167324-5aba-440d-aadd-9bf5344de22e None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:09.846196 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-bd167324-5aba-440d-aadd-9bf5344de22e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:09.885726 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-4d42b5f3-35a1-48ef-a0c1-5e2cbd3033b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3aa3665c65434c7f8dc0c5a9a7590d96'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:09.892895 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-4d42b5f3-35a1-48ef-a0c1-5e2cbd3033b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3aa3665c65434c7f8dc0c5a9a7590d96'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:09.904620 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-4d42b5f3-35a1-48ef-a0c1-5e2cbd3033b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3aa3665c65434c7f8dc0c5a9a7590d96'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:09.905278 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-4d42b5f3-35a1-48ef-a0c1-5e2cbd3033b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3aa3665c65434c7f8dc0c5a9a7590d96'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:09.906014 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-4d42b5f3-35a1-48ef-a0c1-5e2cbd3033b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3aa3665c65434c7f8dc0c5a9a7590d96'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:09.925261 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd167324-5aba-440d-aadd-9bf5344de22e None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:09.930627 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0fc70a-ee4f-4a17-a284-ae07a5ed33f2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c1f878b0dc4d63a4f5dcb12fac1dc1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:09.935770 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0fc70a-ee4f-4a17-a284-ae07a5ed33f2 service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:09.935770 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0fc70a-ee4f-4a17-a284-ae07a5ed33f2 service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:09.935770 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0fc70a-ee4f-4a17-a284-ae07a5ed33f2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:09.935770 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-3a0fc70a-ee4f-4a17-a284-ae07a5ed33f2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:09.944253 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0fc70a-ee4f-4a17-a284-ae07a5ed33f2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:09.945831 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0fc70a-ee4f-4a17-a284-ae07a5ed33f2 service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:09.955536 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 913/1818] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:30:09 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 17 19:30:09.956348 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0fc70a-ee4f-4a17-a284-ae07a5ed33f2 service nova] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:09.957652 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-3a0fc70a-ee4f-4a17-a284-ae07a5ed33f2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:10.029424 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 906/1819] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 17 19:30:09 2022] DELETE /identity/v3/users/3aa3665c65434c7f8dc0c5a9a7590d96 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:30:10.031781 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0fc70a-ee4f-4a17-a284-ae07a5ed33f2 service nova] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:10.086890 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d10836-6b49-4ae1-bc9a-b6ff460f3036 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd05409b38ae346458abfe41ebda1c59e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:10.087724 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d10836-6b49-4ae1-bc9a-b6ff460f3036 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:10.088012 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d10836-6b49-4ae1-bc9a-b6ff460f3036 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:10.088377 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d10836-6b49-4ae1-bc9a-b6ff460f3036 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:10.090704 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b8d10836-6b49-4ae1-bc9a-b6ff460f3036 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:10.201601 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d10836-6b49-4ae1-bc9a-b6ff460f3036 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:10.203041 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d10836-6b49-4ae1-bc9a-b6ff460f3036 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:10.214506 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 914/1820] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:30:09 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:30:10.295780 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d42b5f3-35a1-48ef-a0c1-5e2cbd3033b1 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:10.297223 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-4d42b5f3-35a1-48ef-a0c1-5e2cbd3033b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:10.443386 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d42b5f3-35a1-48ef-a0c1-5e2cbd3033b1 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:10.455047 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f67eb88-ffd5-44b1-8e72-cb53ad971c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:10.456738 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f67eb88-ffd5-44b1-8e72-cb53ad971c82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:10.457875 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f67eb88-ffd5-44b1-8e72-cb53ad971c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:10.458319 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f67eb88-ffd5-44b1-8e72-cb53ad971c82 admin admin] PATH_INFO: `/v3/users/051e3b0893334b8e8f6064bcb266ea64` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:10.460099 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f67eb88-ffd5-44b1-8e72-cb53ad971c82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=051e3b0893334b8e8f6064bcb266ea64)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:10.463728 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f67eb88-ffd5-44b1-8e72-cb53ad971c82 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:10.695235 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-7f67eb88-ffd5-44b1-8e72-cb53ad971c82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '051e3b0893334b8e8f6064bcb266ea64'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:10.706778 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-7f67eb88-ffd5-44b1-8e72-cb53ad971c82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '051e3b0893334b8e8f6064bcb266ea64'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:10.719934 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-7f67eb88-ffd5-44b1-8e72-cb53ad971c82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '051e3b0893334b8e8f6064bcb266ea64'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:10.721184 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-7f67eb88-ffd5-44b1-8e72-cb53ad971c82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '051e3b0893334b8e8f6064bcb266ea64'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:10.722048 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-7f67eb88-ffd5-44b1-8e72-cb53ad971c82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '051e3b0893334b8e8f6064bcb266ea64'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:10.824803 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d10836-6b49-4ae1-bc9a-b6ff460f3036 service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:10.826917 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 907/1821] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 17 19:30:10 2022] DELETE /identity/v3/users/051e3b0893334b8e8f6064bcb266ea64 => generated 0 bytes in 532 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:30:10.827042 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b8d10836-6b49-4ae1-bc9a-b6ff460f3036 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:10.831287 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a81e00b-659d-4980-9a0f-aee42d4d110b None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:10.831438 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a81e00b-659d-4980-9a0f-aee42d4d110b None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:10.833798 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a81e00b-659d-4980-9a0f-aee42d4d110b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:10.852531 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-6a81e00b-659d-4980-9a0f-aee42d4d110b None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:30:10.853642 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-6a81e00b-659d-4980-9a0f-aee42d4d110b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:10.853828 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d10836-6b49-4ae1-bc9a-b6ff460f3036 service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:10.932501 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 908/1822] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:30:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:30:10.936283 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d10b24-36a1-4b2a-bdcf-ab356ff370e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:10.937142 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d10b24-36a1-4b2a-bdcf-ab356ff370e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:10.937429 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d10b24-36a1-4b2a-bdcf-ab356ff370e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:10.937703 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d10b24-36a1-4b2a-bdcf-ab356ff370e6 admin admin] PATH_INFO: `/v3/projects/55fa3d2caf52480c8ecee4c7fd511231` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:10.938916 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d10b24-36a1-4b2a-bdcf-ab356ff370e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=55fa3d2caf52480c8ecee4c7fd511231)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:10.940891 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d10b24-36a1-4b2a-bdcf-ab356ff370e6 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:10.948002 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a81e00b-659d-4980-9a0f-aee42d4d110b None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:10.950539 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-6a81e00b-659d-4980-9a0f-aee42d4d110b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:11.007103 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a81e00b-659d-4980-9a0f-aee42d4d110b None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:11.010688 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee143aa5-dfc1-44b4-8fa9-d3da58c9d5e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:11.011322 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee143aa5-dfc1-44b4-8fa9-d3da58c9d5e1 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:11.011638 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee143aa5-dfc1-44b4-8fa9-d3da58c9d5e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:11.011911 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee143aa5-dfc1-44b4-8fa9-d3da58c9d5e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:11.020802 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee143aa5-dfc1-44b4-8fa9-d3da58c9d5e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:11.022213 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee143aa5-dfc1-44b4-8fa9-d3da58c9d5e1 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:11.032090 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 909/1823] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:30:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:30:11.084495 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-73d10b24-36a1-4b2a-bdcf-ab356ff370e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55fa3d2caf52480c8ecee4c7fd511231', 'request_id': 'req-73d10b24-36a1-4b2a-bdcf-ab356ff370e6'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:11.093947 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-73d10b24-36a1-4b2a-bdcf-ab356ff370e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55fa3d2caf52480c8ecee4c7fd511231', 'request_id': 'req-73d10b24-36a1-4b2a-bdcf-ab356ff370e6'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:11.251921 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-73d10b24-36a1-4b2a-bdcf-ab356ff370e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55fa3d2caf52480c8ecee4c7fd511231', 'request_id': 'req-73d10b24-36a1-4b2a-bdcf-ab356ff370e6'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:11.274807 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-73d10b24-36a1-4b2a-bdcf-ab356ff370e6 admin admin] The token cache is being invalidate because project 55fa3d2caf52480c8ecee4c7fd511231 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 17 19:30:11.275096 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-73d10b24-36a1-4b2a-bdcf-ab356ff370e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:11.275985 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-73d10b24-36a1-4b2a-bdcf-ab356ff370e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:11.277578 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 915/1824] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 17 19:30:10 2022] DELETE /identity/v3/projects/55fa3d2caf52480c8ecee4c7fd511231 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:30:11.283619 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a926389e-e0ff-48cb-aa80-861ab60e0d91 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:11.283987 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a926389e-e0ff-48cb-aa80-861ab60e0d91 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:11.284272 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a926389e-e0ff-48cb-aa80-861ab60e0d91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:11.321610 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-a926389e-e0ff-48cb-aa80-861ab60e0d91 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:30:11.325729 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-a926389e-e0ff-48cb-aa80-861ab60e0d91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:11.466942 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 910/1825] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:30:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:30:11.474995 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4fe766-2ecd-4a3b-a3ed-ae54aa1865c7 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:11.475741 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4fe766-2ecd-4a3b-a3ed-ae54aa1865c7 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:11.476566 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4fe766-2ecd-4a3b-a3ed-ae54aa1865c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:11.508947 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-5c4fe766-2ecd-4a3b-a3ed-ae54aa1865c7 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:30:11.510512 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-5c4fe766-2ecd-4a3b-a3ed-ae54aa1865c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:11.523800 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 916/1826] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:30:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:30:11.527892 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a926389e-e0ff-48cb-aa80-861ab60e0d91 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:11.539568 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a926389e-e0ff-48cb-aa80-861ab60e0d91 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:11.544597 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529755b5-c207-40ff-8ae8-e1ff630c6113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:11.545252 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529755b5-c207-40ff-8ae8-e1ff630c6113 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:11.545561 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529755b5-c207-40ff-8ae8-e1ff630c6113 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:11.550517 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529755b5-c207-40ff-8ae8-e1ff630c6113 admin admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:11.550517 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529755b5-c207-40ff-8ae8-e1ff630c6113 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:30:11.550517 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529755b5-c207-40ff-8ae8-e1ff630c6113 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:11.550517 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529755b5-c207-40ff-8ae8-e1ff630c6113 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:11.561885 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 911/1827] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 17 19:30:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:30:11.566095 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4fe766-2ecd-4a3b-a3ed-ae54aa1865c7 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:11.593477 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4fe766-2ecd-4a3b-a3ed-ae54aa1865c7 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:11.598403 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5ef8ac-a0cc-4840-878d-bbbdd7ea8a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:11.602203 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5ef8ac-a0cc-4840-878d-bbbdd7ea8a41 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:11.602203 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5ef8ac-a0cc-4840-878d-bbbdd7ea8a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:11.602203 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5ef8ac-a0cc-4840-878d-bbbdd7ea8a41 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:11.602203 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5ef8ac-a0cc-4840-878d-bbbdd7ea8a41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:11.602615 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5ef8ac-a0cc-4840-878d-bbbdd7ea8a41 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:11.691986 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529755b5-c207-40ff-8ae8-e1ff630c6113 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:11.696794 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-529755b5-c207-40ff-8ae8-e1ff630c6113 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:11.699105 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 917/1828] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 17 19:30:11 2022] POST /identity/v3/projects => generated 385 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 17 19:30:11.707864 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5ef8ac-a0cc-4840-878d-bbbdd7ea8a41 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:11.726250 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529755b5-c207-40ff-8ae8-e1ff630c6113 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:11.783084 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5ef8ac-a0cc-4840-878d-bbbdd7ea8a41 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:11.788028 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcaf2ad-5107-47d3-a54d-0fec2083582e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:11.788703 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcaf2ad-5107-47d3-a54d-0fec2083582e admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:11.788984 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcaf2ad-5107-47d3-a54d-0fec2083582e admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:11.789481 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcaf2ad-5107-47d3-a54d-0fec2083582e admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:11.789993 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcaf2ad-5107-47d3-a54d-0fec2083582e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:11.792068 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcaf2ad-5107-47d3-a54d-0fec2083582e admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:11.818169 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba637f35-3c7c-47c2-a5e3-c1c253ed820f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:11.819094 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba637f35-3c7c-47c2-a5e3-c1c253ed820f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:11.819377 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba637f35-3c7c-47c2-a5e3-c1c253ed820f admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:11.819689 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba637f35-3c7c-47c2-a5e3-c1c253ed820f admin admin] PATH_INFO: `/v3/projects/d758604b15e942098679fcdf859a699c` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:11.820947 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba637f35-3c7c-47c2-a5e3-c1c253ed820f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d758604b15e942098679fcdf859a699c)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:11.826442 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba637f35-3c7c-47c2-a5e3-c1c253ed820f admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:11.929564 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 918/1829] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 17 19:30:11 2022] POST /identity/v3/users => generated 293 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:30:11.933607 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcaf2ad-5107-47d3-a54d-0fec2083582e admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:11.943124 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcaf2ad-5107-47d3-a54d-0fec2083582e admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:11.949045 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f45116-633b-4b3d-b83d-1879b3f7576c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:11.949776 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f45116-633b-4b3d-b83d-1879b3f7576c admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:11.949854 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f45116-633b-4b3d-b83d-1879b3f7576c admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:11.950131 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f45116-633b-4b3d-b83d-1879b3f7576c admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:11.951424 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f45116-633b-4b3d-b83d-1879b3f7576c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:11.953060 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f45116-633b-4b3d-b83d-1879b3f7576c admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:11.963201 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 919/1830] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:30:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 17 19:30:11.967666 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f45116-633b-4b3d-b83d-1879b3f7576c admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:11.977243 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f45116-633b-4b3d-b83d-1879b3f7576c admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:11.981998 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c649a6ac-8482-47a4-a350-84cb73c5df76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:11.982634 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c649a6ac-8482-47a4-a350-84cb73c5df76 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:11.982915 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c649a6ac-8482-47a4-a350-84cb73c5df76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:11.983182 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c649a6ac-8482-47a4-a350-84cb73c5df76 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:11.983911 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c649a6ac-8482-47a4-a350-84cb73c5df76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:11.985499 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c649a6ac-8482-47a4-a350-84cb73c5df76 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:11.994373 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-ba637f35-3c7c-47c2-a5e3-c1c253ed820f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd758604b15e942098679fcdf859a699c', 'request_id': 'req-ba637f35-3c7c-47c2-a5e3-c1c253ed820f'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:11.995837 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 920/1831] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:30:11 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 17 19:30:12.000531 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c649a6ac-8482-47a4-a350-84cb73c5df76 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:12.003696 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-ba637f35-3c7c-47c2-a5e3-c1c253ed820f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd758604b15e942098679fcdf859a699c', 'request_id': 'req-ba637f35-3c7c-47c2-a5e3-c1c253ed820f'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:12.006610 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-ba637f35-3c7c-47c2-a5e3-c1c253ed820f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd758604b15e942098679fcdf859a699c', 'request_id': 'req-ba637f35-3c7c-47c2-a5e3-c1c253ed820f'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:12.010222 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c649a6ac-8482-47a4-a350-84cb73c5df76 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:12.065114 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ad6b6d-6587-4dad-b131-4b6448a2bed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:12.066003 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ad6b6d-6587-4dad-b131-4b6448a2bed9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:12.066284 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ad6b6d-6587-4dad-b131-4b6448a2bed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:12.066557 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ad6b6d-6587-4dad-b131-4b6448a2bed9 admin admin] PATH_INFO: `/v3/projects/d3d28e72f0c84bc0960a840bf11d33a2/users/06038d0e657d42aa9c3c2771b4d2c601/roles/114e6a16e63447d1924707ace5aace25` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:12.082096 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ad6b6d-6587-4dad-b131-4b6448a2bed9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3d28e72f0c84bc0960a840bf11d33a2, user_id=06038d0e657d42aa9c3c2771b4d2c601, role_id=114e6a16e63447d1924707ace5aace25)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:12.084782 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ad6b6d-6587-4dad-b131-4b6448a2bed9 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:12.125800 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-ba637f35-3c7c-47c2-a5e3-c1c253ed820f admin admin] The token cache is being invalidate because project d758604b15e942098679fcdf859a699c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 17 19:30:12.126091 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-ba637f35-3c7c-47c2-a5e3-c1c253ed820f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:12.126734 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-ba637f35-3c7c-47c2-a5e3-c1c253ed820f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:12.128364 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 912/1832] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 17 19:30:11 2022] DELETE /identity/v3/projects/d758604b15e942098679fcdf859a699c => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:30:12.133066 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e502282d-2f60-4ab5-b5be-80557fc2d272 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:12.133559 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e502282d-2f60-4ab5-b5be-80557fc2d272 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:12.133822 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e502282d-2f60-4ab5-b5be-80557fc2d272 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:12.152551 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-e502282d-2f60-4ab5-b5be-80557fc2d272 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:30:12.153895 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-e502282d-2f60-4ab5-b5be-80557fc2d272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:12.167608 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 913/1833] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:30:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 17 19:30:12.170591 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e3fc72-a47e-451d-b491-685e4a6bd220 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:12.170905 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e3fc72-a47e-451d-b491-685e4a6bd220 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:12.171180 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e3fc72-a47e-451d-b491-685e4a6bd220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:12.190823 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-a7e3fc72-a47e-451d-b491-685e4a6bd220 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:30:12.191986 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-a7e3fc72-a47e-451d-b491-685e4a6bd220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:12.200124 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 914/1834] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:30:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 17 19:30:12.204796 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e3fc72-a47e-451d-b491-685e4a6bd220 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:12.213476 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e3fc72-a47e-451d-b491-685e4a6bd220 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:12.217967 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f8bebb-857a-439d-8742-eda21a0c0d3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:12.218606 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f8bebb-857a-439d-8742-eda21a0c0d3d admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:12.218882 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f8bebb-857a-439d-8742-eda21a0c0d3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:12.219149 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f8bebb-857a-439d-8742-eda21a0c0d3d admin admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:12.219898 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f8bebb-857a-439d-8742-eda21a0c0d3d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:30:12.220196 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f8bebb-857a-439d-8742-eda21a0c0d3d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:12.222205 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f8bebb-857a-439d-8742-eda21a0c0d3d admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:12.239507 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 915/1835] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 17 19:30:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:30:12.240392 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f8bebb-857a-439d-8742-eda21a0c0d3d admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:12.241901 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 921/1836] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:30:11 2022] PUT /identity/v3/projects/d3d28e72f0c84bc0960a840bf11d33a2/users/06038d0e657d42aa9c3c2771b4d2c601/roles/114e6a16e63447d1924707ace5aace25 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 17 19:30:12.250594 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f8bebb-857a-439d-8742-eda21a0c0d3d admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:12.253816 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1990f7ee-60f2-4012-86ca-201dfb1639e5 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:12.254313 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1990f7ee-60f2-4012-86ca-201dfb1639e5 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:12.254657 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1990f7ee-60f2-4012-86ca-201dfb1639e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:12.291420 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-1990f7ee-60f2-4012-86ca-201dfb1639e5 None None] MFA Rules not processed for user `06038d0e657d42aa9c3c2771b4d2c601`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:30:12.292693 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-1990f7ee-60f2-4012-86ca-201dfb1639e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:12.329141 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5f8007-c965-43c1-a277-56fc81e87dce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:12.329842 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5f8007-c965-43c1-a277-56fc81e87dce admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:12.330137 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5f8007-c965-43c1-a277-56fc81e87dce admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:12.330418 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5f8007-c965-43c1-a277-56fc81e87dce admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:12.331398 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5f8007-c965-43c1-a277-56fc81e87dce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:12.334703 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5f8007-c965-43c1-a277-56fc81e87dce admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:12.361340 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 922/1837] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:30:12 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:30:12.366117 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1990f7ee-60f2-4012-86ca-201dfb1639e5 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:12.371921 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-1990f7ee-60f2-4012-86ca-201dfb1639e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:12.392229 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1990f7ee-60f2-4012-86ca-201dfb1639e5 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:12.397416 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be0d2bc-be4a-4505-807c-4bd670bc50e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:12.398412 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be0d2bc-be4a-4505-807c-4bd670bc50e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:12.398687 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be0d2bc-be4a-4505-807c-4bd670bc50e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:12.398955 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be0d2bc-be4a-4505-807c-4bd670bc50e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:12.399697 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be0d2bc-be4a-4505-807c-4bd670bc50e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:12.401905 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be0d2bc-be4a-4505-807c-4bd670bc50e7 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:12.465045 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 916/1838] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 17 19:30:12 2022] POST /identity/v3/projects => generated 363 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:30:12.469334 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5f8007-c965-43c1-a277-56fc81e87dce admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:12.510177 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 923/1839] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 17 19:30:12 2022] POST /identity/v3/projects => generated 385 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:30:12.528595 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be0d2bc-be4a-4505-807c-4bd670bc50e7 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:12.699581 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be0d2bc-be4a-4505-807c-4bd670bc50e7 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:12.718919 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4f5983-2a3b-466c-af1d-938684e1ab6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:12.721635 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4f5983-2a3b-466c-af1d-938684e1ab6c admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:12.724244 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4f5983-2a3b-466c-af1d-938684e1ab6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:12.724244 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4f5983-2a3b-466c-af1d-938684e1ab6c admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:12.727055 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4f5983-2a3b-466c-af1d-938684e1ab6c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:12.734449 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4f5983-2a3b-466c-af1d-938684e1ab6c admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:12.735025 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5f8007-c965-43c1-a277-56fc81e87dce admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:12.756722 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb16563-a1a8-4dff-8b46-4a4c603200f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:12.757389 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb16563-a1a8-4dff-8b46-4a4c603200f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:12.770385 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb16563-a1a8-4dff-8b46-4a4c603200f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:12.770385 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb16563-a1a8-4dff-8b46-4a4c603200f7 admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:12.770385 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb16563-a1a8-4dff-8b46-4a4c603200f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:12.770385 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb16563-a1a8-4dff-8b46-4a4c603200f7 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:12.889033 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 924/1840] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 17 19:30:12 2022] POST /identity/v3/users => generated 299 bytes in 358 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:30:12.890269 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4f5983-2a3b-466c-af1d-938684e1ab6c admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:12.905316 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4f5983-2a3b-466c-af1d-938684e1ab6c admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:12.914402 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53441c40-6860-4e32-8d3d-f6b0c0d6bd3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:12.915222 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53441c40-6860-4e32-8d3d-f6b0c0d6bd3c admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:12.915433 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53441c40-6860-4e32-8d3d-f6b0c0d6bd3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:12.915873 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53441c40-6860-4e32-8d3d-f6b0c0d6bd3c admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:12.915971 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 917/1841] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 17 19:30:12 2022] POST /identity/v3/users => generated 282 bytes in 447 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:30:12.917055 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53441c40-6860-4e32-8d3d-f6b0c0d6bd3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:12.918958 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53441c40-6860-4e32-8d3d-f6b0c0d6bd3c admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:12.920584 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb16563-a1a8-4dff-8b46-4a4c603200f7 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:12.948464 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 925/1842] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:30:12 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:30:12.951356 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53441c40-6860-4e32-8d3d-f6b0c0d6bd3c admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:12.952397 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb16563-a1a8-4dff-8b46-4a4c603200f7 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:12.959435 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-53441c40-6860-4e32-8d3d-f6b0c0d6bd3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:12.962924 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04160791-f5dc-48d1-8066-1cd78dd63be7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:12.963640 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04160791-f5dc-48d1-8066-1cd78dd63be7 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:12.963941 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04160791-f5dc-48d1-8066-1cd78dd63be7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:12.964207 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04160791-f5dc-48d1-8066-1cd78dd63be7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:12.964910 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04160791-f5dc-48d1-8066-1cd78dd63be7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:12.967090 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04160791-f5dc-48d1-8066-1cd78dd63be7 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:12.994584 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53441c40-6860-4e32-8d3d-f6b0c0d6bd3c admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:12.998192 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 918/1843] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:30:12 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:30:12.999080 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04160791-f5dc-48d1-8066-1cd78dd63be7 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:13.013854 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0994f9-3e6e-4b40-8efb-55e51ec5d63f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:13.014927 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0994f9-3e6e-4b40-8efb-55e51ec5d63f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:13.015218 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0994f9-3e6e-4b40-8efb-55e51ec5d63f admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:13.015651 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0994f9-3e6e-4b40-8efb-55e51ec5d63f admin admin] PATH_INFO: `/v3/users/d78743112b7b4e6f90dab0daa6268664` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:13.019368 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0994f9-3e6e-4b40-8efb-55e51ec5d63f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d78743112b7b4e6f90dab0daa6268664)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:13.021352 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0994f9-3e6e-4b40-8efb-55e51ec5d63f admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:13.022217 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04160791-f5dc-48d1-8066-1cd78dd63be7 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:13.027786 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5745a008-5bf2-46dc-bf2c-7777d0d8bac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:13.029341 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5745a008-5bf2-46dc-bf2c-7777d0d8bac9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:13.029825 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5745a008-5bf2-46dc-bf2c-7777d0d8bac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:13.030099 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5745a008-5bf2-46dc-bf2c-7777d0d8bac9 admin admin] PATH_INFO: `/v3/projects/f23d43ffed3d43b2baa79d453c5c2f91/users/2cf4daf86ed448868c36a8fec2a5ac6b/roles/60af5088793e4379b88821a86552a344` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:13.051586 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5745a008-5bf2-46dc-bf2c-7777d0d8bac9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f23d43ffed3d43b2baa79d453c5c2f91, user_id=2cf4daf86ed448868c36a8fec2a5ac6b, role_id=60af5088793e4379b88821a86552a344)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:13.054384 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5745a008-5bf2-46dc-bf2c-7777d0d8bac9 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:13.223566 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 919/1844] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:30:12 2022] PUT /identity/v3/projects/f23d43ffed3d43b2baa79d453c5c2f91/users/2cf4daf86ed448868c36a8fec2a5ac6b/roles/60af5088793e4379b88821a86552a344 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:30:13.224971 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5745a008-5bf2-46dc-bf2c-7777d0d8bac9 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:13.291958 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5745a008-5bf2-46dc-bf2c-7777d0d8bac9 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:13.298496 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7b4fd5-f491-4b99-8f01-8b9d6143cee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:13.299162 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7b4fd5-f491-4b99-8f01-8b9d6143cee3 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:13.299437 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7b4fd5-f491-4b99-8f01-8b9d6143cee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:13.299737 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7b4fd5-f491-4b99-8f01-8b9d6143cee3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:13.300461 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7b4fd5-f491-4b99-8f01-8b9d6143cee3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:13.302036 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7b4fd5-f491-4b99-8f01-8b9d6143cee3 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:13.313644 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 920/1845] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:30:13 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:30:13.317996 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7b4fd5-f491-4b99-8f01-8b9d6143cee3 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:13.325217 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7b4fd5-f491-4b99-8f01-8b9d6143cee3 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:13.330380 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bb2d75-a8a3-4271-add2-39b27b3088dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:13.331143 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bb2d75-a8a3-4271-add2-39b27b3088dd admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:13.331668 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bb2d75-a8a3-4271-add2-39b27b3088dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:13.331940 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bb2d75-a8a3-4271-add2-39b27b3088dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:13.332654 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bb2d75-a8a3-4271-add2-39b27b3088dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:13.334649 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bb2d75-a8a3-4271-add2-39b27b3088dd admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:13.344913 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 921/1846] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:30:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:30:13.345729 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bb2d75-a8a3-4271-add2-39b27b3088dd admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:13.358996 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bb2d75-a8a3-4271-add2-39b27b3088dd admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:13.366266 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f9d0de-558c-4bc5-a34f-2713df3a14fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:13.368288 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f9d0de-558c-4bc5-a34f-2713df3a14fa admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:13.368288 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f9d0de-558c-4bc5-a34f-2713df3a14fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:13.368288 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f9d0de-558c-4bc5-a34f-2713df3a14fa admin admin] PATH_INFO: `/v3/projects/28807197c3334ba5b24736fa2b50f9f3/users/3bad77b438024885a54e0de1b0d821b1/roles/114e6a16e63447d1924707ace5aace25` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:13.384317 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f9d0de-558c-4bc5-a34f-2713df3a14fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=28807197c3334ba5b24736fa2b50f9f3, user_id=3bad77b438024885a54e0de1b0d821b1, role_id=114e6a16e63447d1924707ace5aace25)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:13.387069 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f9d0de-558c-4bc5-a34f-2713df3a14fa admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:13.444488 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-fa0994f9-3e6e-4b40-8efb-55e51ec5d63f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd78743112b7b4e6f90dab0daa6268664'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:13.445448 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-fa0994f9-3e6e-4b40-8efb-55e51ec5d63f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd78743112b7b4e6f90dab0daa6268664'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:13.458022 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-fa0994f9-3e6e-4b40-8efb-55e51ec5d63f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd78743112b7b4e6f90dab0daa6268664'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:13.538557 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-fa0994f9-3e6e-4b40-8efb-55e51ec5d63f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd78743112b7b4e6f90dab0daa6268664'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:13.545656 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 922/1847] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:30:13 2022] PUT /identity/v3/projects/28807197c3334ba5b24736fa2b50f9f3/users/3bad77b438024885a54e0de1b0d821b1/roles/114e6a16e63447d1924707ace5aace25 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:30:13.545966 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f9d0de-558c-4bc5-a34f-2713df3a14fa admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:13.553566 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-34f9d0de-558c-4bc5-a34f-2713df3a14fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:13.556574 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-fa0994f9-3e6e-4b40-8efb-55e51ec5d63f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd78743112b7b4e6f90dab0daa6268664'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:13.567808 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 926/1848] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 17 19:30:12 2022] DELETE /identity/v3/users/d78743112b7b4e6f90dab0daa6268664 => generated 0 bytes in 619 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:30:13.569508 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b520c6-5ae5-47d0-8b61-11e7da5f3892 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:13.569801 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b520c6-5ae5-47d0-8b61-11e7da5f3892 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:13.570074 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b520c6-5ae5-47d0-8b61-11e7da5f3892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:13.606868 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-42b520c6-5ae5-47d0-8b61-11e7da5f3892 None None] MFA Rules not processed for user `3bad77b438024885a54e0de1b0d821b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:30:13.608107 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-42b520c6-5ae5-47d0-8b61-11e7da5f3892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:13.684089 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 927/1849] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:30:13 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:30:13.686233 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b76b81-3d6e-4be4-9350-9e1d912f2101 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:13.686520 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b76b81-3d6e-4be4-9350-9e1d912f2101 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:13.687520 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b76b81-3d6e-4be4-9350-9e1d912f2101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:13.712199 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-18b76b81-3d6e-4be4-9350-9e1d912f2101 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:30:13.713377 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-18b76b81-3d6e-4be4-9350-9e1d912f2101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:13.723275 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f9d0de-558c-4bc5-a34f-2713df3a14fa admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:13.832589 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ae0674-c7a9-4d21-8fd1-40460f26ffb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:13.837077 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ae0674-c7a9-4d21-8fd1-40460f26ffb0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:13.838888 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ae0674-c7a9-4d21-8fd1-40460f26ffb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:13.839578 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ae0674-c7a9-4d21-8fd1-40460f26ffb0 admin admin] PATH_INFO: `/v3/projects/f23d43ffed3d43b2baa79d453c5c2f91/users/2cf4daf86ed448868c36a8fec2a5ac6b/roles/60af5088793e4379b88821a86552a344` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:13.850306 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ae0674-c7a9-4d21-8fd1-40460f26ffb0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f23d43ffed3d43b2baa79d453c5c2f91, user_id=2cf4daf86ed448868c36a8fec2a5ac6b, role_id=60af5088793e4379b88821a86552a344)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:13.855325 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ae0674-c7a9-4d21-8fd1-40460f26ffb0 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:13.864707 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 928/1850] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:30:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:30:13.876104 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b76b81-3d6e-4be4-9350-9e1d912f2101 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:13.880270 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-18b76b81-3d6e-4be4-9350-9e1d912f2101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:13.894134 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 923/1851] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:30:13 2022] PUT /identity/v3/projects/f23d43ffed3d43b2baa79d453c5c2f91/users/2cf4daf86ed448868c36a8fec2a5ac6b/roles/60af5088793e4379b88821a86552a344 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:30:13.904278 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ae0674-c7a9-4d21-8fd1-40460f26ffb0 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:13.907669 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-67ae0674-c7a9-4d21-8fd1-40460f26ffb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:13.907936 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b76b81-3d6e-4be4-9350-9e1d912f2101 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:14.018287 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c48608-0dc5-43c0-804d-a2831309db9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:14.019726 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c48608-0dc5-43c0-804d-a2831309db9a admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:14.020065 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c48608-0dc5-43c0-804d-a2831309db9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:14.020832 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c48608-0dc5-43c0-804d-a2831309db9a admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:14.024049 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c48608-0dc5-43c0-804d-a2831309db9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:14.032139 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c48608-0dc5-43c0-804d-a2831309db9a admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:14.049780 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ae0674-c7a9-4d21-8fd1-40460f26ffb0 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:14.063359 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2db126c-3b8d-4f93-b619-a9e7406f7605 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:14.069754 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2db126c-3b8d-4f93-b619-a9e7406f7605 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:14.070300 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2db126c-3b8d-4f93-b619-a9e7406f7605 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:14.071815 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2db126c-3b8d-4f93-b619-a9e7406f7605 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:14.100491 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2db126c-3b8d-4f93-b619-a9e7406f7605 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:14.105742 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2db126c-3b8d-4f93-b619-a9e7406f7605 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:14.136265 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 924/1852] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:30:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:30:14.137744 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f97af37-f5b6-46bb-8596-2badbc84c60a None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:14.138657 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f97af37-f5b6-46bb-8596-2badbc84c60a None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:14.138988 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f97af37-f5b6-46bb-8596-2badbc84c60a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:14.176547 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 929/1853] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 17 19:30:13 2022] POST /identity/v3/projects => generated 365 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:30:14.185648 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c48608-0dc5-43c0-804d-a2831309db9a admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:14.275386 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-8f97af37-f5b6-46bb-8596-2badbc84c60a None None] MFA Rules not processed for user `2cf4daf86ed448868c36a8fec2a5ac6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:30:14.282365 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-8f97af37-f5b6-46bb-8596-2badbc84c60a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:14.316471 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c48608-0dc5-43c0-804d-a2831309db9a admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:14.330832 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae499ed-37c3-4372-8515-fff07ff1b648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:14.333349 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae499ed-37c3-4372-8515-fff07ff1b648 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:14.334595 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae499ed-37c3-4372-8515-fff07ff1b648 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:14.336913 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae499ed-37c3-4372-8515-fff07ff1b648 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:14.342654 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae499ed-37c3-4372-8515-fff07ff1b648 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:14.356209 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae499ed-37c3-4372-8515-fff07ff1b648 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:14.460369 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 925/1854] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:30:14 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:30:14.462837 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a1491a-3e19-4bee-9fa9-b6dba9740ee5 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:14.463319 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a1491a-3e19-4bee-9fa9-b6dba9740ee5 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:14.463816 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a1491a-3e19-4bee-9fa9-b6dba9740ee5 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:14.473857 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 926/1855] 10.222.0.58 () {54 vars in 921 bytes} [Tue May 17 19:30:14 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:30:14.479350 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a1491a-3e19-4bee-9fa9-b6dba9740ee5 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:14.480775 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-81a1491a-3e19-4bee-9fa9-b6dba9740ee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:14.521837 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 930/1856] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 17 19:30:14 2022] POST /identity/v3/users => generated 289 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:30:14.529164 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa108b12-5137-451c-a523-4fb13cbc4258 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:14.529460 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa108b12-5137-451c-a523-4fb13cbc4258 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:14.529783 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa108b12-5137-451c-a523-4fb13cbc4258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:14.595626 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a1491a-3e19-4bee-9fa9-b6dba9740ee5 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:14.601561 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-fa108b12-5137-451c-a523-4fb13cbc4258 None None] MFA Rules not processed for user `65c1f878b0dc4d63a4f5dcb12fac1dc1`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:30:14.602991 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-fa108b12-5137-451c-a523-4fb13cbc4258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:14.608309 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c4b3c5-9680-4e47-9c00-0d1524a3711c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd05409b38ae346458abfe41ebda1c59e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:14.612373 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c4b3c5-9680-4e47-9c00-0d1524a3711c service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:14.613791 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c4b3c5-9680-4e47-9c00-0d1524a3711c service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:14.615526 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c4b3c5-9680-4e47-9c00-0d1524a3711c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:14.629935 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c4b3c5-9680-4e47-9c00-0d1524a3711c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:14.632801 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c4b3c5-9680-4e47-9c00-0d1524a3711c service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:14.658833 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 927/1857] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:30:14 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:30:14.698674 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c4b3c5-9680-4e47-9c00-0d1524a3711c service cinder] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:14.746102 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 931/1858] 10.222.0.58 () {58 vars in 1018 bytes} [Tue May 17 19:30:14 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:30:14.753082 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c4b3c5-9680-4e47-9c00-0d1524a3711c service cinder] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:14.761266 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa108b12-5137-451c-a523-4fb13cbc4258 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:14.764758 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-fa108b12-5137-451c-a523-4fb13cbc4258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:14.773358 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f369669-d3d9-4108-b43f-3f2f7e64eb2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:14.773358 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f369669-d3d9-4108-b43f-3f2f7e64eb2f admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:14.773358 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f369669-d3d9-4108-b43f-3f2f7e64eb2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:14.773594 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f369669-d3d9-4108-b43f-3f2f7e64eb2f admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:14.776408 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f369669-d3d9-4108-b43f-3f2f7e64eb2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:14.778027 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f369669-d3d9-4108-b43f-3f2f7e64eb2f admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:14.804152 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 928/1859] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:30:14 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:30:14.806367 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa108b12-5137-451c-a523-4fb13cbc4258 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:14.808509 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f369669-d3d9-4108-b43f-3f2f7e64eb2f admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:14.824903 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f369669-d3d9-4108-b43f-3f2f7e64eb2f admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:14.839859 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bd8711-6476-47b4-ab52-a5d42f6b5b92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:14.840812 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bd8711-6476-47b4-ab52-a5d42f6b5b92 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:14.841782 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bd8711-6476-47b4-ab52-a5d42f6b5b92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:14.842070 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bd8711-6476-47b4-ab52-a5d42f6b5b92 admin admin] PATH_INFO: `/v3/projects/ba1c1e27ce5c437f8496cf330abf9030/users/d4c8804ee59340cc84367cdbe9f21967/roles/60af5088793e4379b88821a86552a344` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:14.847975 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbe2b86-dd5b-47fb-9aaf-31366cb5f68c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c1f878b0dc4d63a4f5dcb12fac1dc1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:14.848658 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbe2b86-dd5b-47fb-9aaf-31366cb5f68c service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:14.848934 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbe2b86-dd5b-47fb-9aaf-31366cb5f68c service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:14.849205 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbe2b86-dd5b-47fb-9aaf-31366cb5f68c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:14.859156 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbe2b86-dd5b-47fb-9aaf-31366cb5f68c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:14.863006 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbe2b86-dd5b-47fb-9aaf-31366cb5f68c service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:14.870084 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bd8711-6476-47b4-ab52-a5d42f6b5b92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba1c1e27ce5c437f8496cf330abf9030, user_id=d4c8804ee59340cc84367cdbe9f21967, role_id=60af5088793e4379b88821a86552a344)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:14.873195 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bd8711-6476-47b4-ab52-a5d42f6b5b92 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:14.884582 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 932/1860] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:30:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:30:14.885776 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbe2b86-dd5b-47fb-9aaf-31366cb5f68c service nova] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:14.904606 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbe2b86-dd5b-47fb-9aaf-31366cb5f68c service nova] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:14.910769 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1603fe38-62f8-498b-82e3-adae80c5377a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:14.911952 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1603fe38-62f8-498b-82e3-adae80c5377a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:14.912294 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1603fe38-62f8-498b-82e3-adae80c5377a admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:14.912808 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1603fe38-62f8-498b-82e3-adae80c5377a admin admin] PATH_INFO: `/v3/users/06038d0e657d42aa9c3c2771b4d2c601` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:14.914323 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1603fe38-62f8-498b-82e3-adae80c5377a admin admin] RBAC: Authorizing `identity:delete_user(user_id=06038d0e657d42aa9c3c2771b4d2c601)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:14.916984 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1603fe38-62f8-498b-82e3-adae80c5377a admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:14.973847 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 929/1861] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:30:14 2022] PUT /identity/v3/projects/ba1c1e27ce5c437f8496cf330abf9030/users/d4c8804ee59340cc84367cdbe9f21967/roles/60af5088793e4379b88821a86552a344 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 17 19:30:14.974903 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bd8711-6476-47b4-ab52-a5d42f6b5b92 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:15.078021 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bd8711-6476-47b4-ab52-a5d42f6b5b92 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:15.082972 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f08022a-53c6-45c5-8c73-09d11741e8b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:15.083836 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f08022a-53c6-45c5-8c73-09d11741e8b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:15.084124 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f08022a-53c6-45c5-8c73-09d11741e8b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:15.084402 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f08022a-53c6-45c5-8c73-09d11741e8b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:15.085132 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f08022a-53c6-45c5-8c73-09d11741e8b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:15.086851 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f08022a-53c6-45c5-8c73-09d11741e8b1 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:15.099901 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 930/1862] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:30:14 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 17 19:30:15.103830 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f08022a-53c6-45c5-8c73-09d11741e8b1 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:15.121148 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f08022a-53c6-45c5-8c73-09d11741e8b1 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:15.127635 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e2a895-7484-4417-9a54-95bf7605483e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:15.128740 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e2a895-7484-4417-9a54-95bf7605483e admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:15.129067 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e2a895-7484-4417-9a54-95bf7605483e admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:15.129341 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e2a895-7484-4417-9a54-95bf7605483e admin admin] PATH_INFO: `/v3/projects/ba1c1e27ce5c437f8496cf330abf9030/users/d4c8804ee59340cc84367cdbe9f21967/roles/60af5088793e4379b88821a86552a344` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:15.131399 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e2a895-7484-4417-9a54-95bf7605483e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba1c1e27ce5c437f8496cf330abf9030, user_id=d4c8804ee59340cc84367cdbe9f21967, role_id=60af5088793e4379b88821a86552a344)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:15.135076 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e2a895-7484-4417-9a54-95bf7605483e admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:15.148287 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 931/1863] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:30:15 2022] PUT /identity/v3/projects/ba1c1e27ce5c437f8496cf330abf9030/users/d4c8804ee59340cc84367cdbe9f21967/roles/60af5088793e4379b88821a86552a344 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 17 19:30:15.148287 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e2a895-7484-4417-9a54-95bf7605483e admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:15.161160 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-48e2a895-7484-4417-9a54-95bf7605483e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:15.186360 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1603fe38-62f8-498b-82e3-adae80c5377a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06038d0e657d42aa9c3c2771b4d2c601'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:15.187146 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1603fe38-62f8-498b-82e3-adae80c5377a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06038d0e657d42aa9c3c2771b4d2c601'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:15.213844 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1603fe38-62f8-498b-82e3-adae80c5377a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06038d0e657d42aa9c3c2771b4d2c601'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:15.263740 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e2a895-7484-4417-9a54-95bf7605483e admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:15.271292 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c311927-e01c-4ea4-9875-0863577e198d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:15.272523 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c311927-e01c-4ea4-9875-0863577e198d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:15.273061 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c311927-e01c-4ea4-9875-0863577e198d admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:15.273701 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c311927-e01c-4ea4-9875-0863577e198d admin admin] PATH_INFO: `/v3/projects/32020b38b382462d8e702c42972d2d1d` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:15.275385 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c311927-e01c-4ea4-9875-0863577e198d admin admin] RBAC: Authorizing `identity:delete_project(project_id=32020b38b382462d8e702c42972d2d1d)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:15.278173 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c311927-e01c-4ea4-9875-0863577e198d admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:15.302693 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1603fe38-62f8-498b-82e3-adae80c5377a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06038d0e657d42aa9c3c2771b4d2c601'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:15.319606 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1603fe38-62f8-498b-82e3-adae80c5377a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06038d0e657d42aa9c3c2771b4d2c601'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:15.350616 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 933/1864] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 17 19:30:14 2022] DELETE /identity/v3/users/06038d0e657d42aa9c3c2771b4d2c601 => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:30:15.352301 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b39d914-c207-486e-b60d-bc3f8b34afa8 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:15.353373 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b39d914-c207-486e-b60d-bc3f8b34afa8 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:15.353772 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b39d914-c207-486e-b60d-bc3f8b34afa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:15.409749 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-1b39d914-c207-486e-b60d-bc3f8b34afa8 None None] MFA Rules not processed for user `d4c8804ee59340cc84367cdbe9f21967`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:30:15.411399 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-1b39d914-c207-486e-b60d-bc3f8b34afa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:15.445237 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-6c311927-e01c-4ea4-9875-0863577e198d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32020b38b382462d8e702c42972d2d1d', 'request_id': 'req-6c311927-e01c-4ea4-9875-0863577e198d'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:15.469784 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-6c311927-e01c-4ea4-9875-0863577e198d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32020b38b382462d8e702c42972d2d1d', 'request_id': 'req-6c311927-e01c-4ea4-9875-0863577e198d'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:15.485667 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-6c311927-e01c-4ea4-9875-0863577e198d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32020b38b382462d8e702c42972d2d1d', 'request_id': 'req-6c311927-e01c-4ea4-9875-0863577e198d'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:15.514340 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 934/1865] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:30:15 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:30:15.515981 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f66536-eeb3-40f7-8696-652953baf4fc None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:15.516533 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f66536-eeb3-40f7-8696-652953baf4fc None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:15.516533 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f66536-eeb3-40f7-8696-652953baf4fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:15.543600 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-09f66536-eeb3-40f7-8696-652953baf4fc None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:30:15.545653 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-09f66536-eeb3-40f7-8696-652953baf4fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:15.620803 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 935/1866] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:30:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:30:15.623192 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-6c311927-e01c-4ea4-9875-0863577e198d admin admin] The token cache is being invalidate because project 32020b38b382462d8e702c42972d2d1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 17 19:30:15.623291 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f66536-eeb3-40f7-8696-652953baf4fc None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:15.623364 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-6c311927-e01c-4ea4-9875-0863577e198d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:15.624605 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-6c311927-e01c-4ea4-9875-0863577e198d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:15.626487 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-09f66536-eeb3-40f7-8696-652953baf4fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:15.627123 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 932/1867] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 17 19:30:15 2022] DELETE /identity/v3/projects/32020b38b382462d8e702c42972d2d1d => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:30:15.630948 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c311927-e01c-4ea4-9875-0863577e198d admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:15.632402 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-6c311927-e01c-4ea4-9875-0863577e198d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:15.650363 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c311927-e01c-4ea4-9875-0863577e198d admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:15.656728 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0702ba4e-e841-41ff-b150-0d30adf8aa49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:15.658250 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0702ba4e-e841-41ff-b150-0d30adf8aa49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:15.658782 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0702ba4e-e841-41ff-b150-0d30adf8aa49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:15.659349 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0702ba4e-e841-41ff-b150-0d30adf8aa49 admin admin] PATH_INFO: `/v3/users/6cdfe21105c74d4299e7fe2f9b4ac23d` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:15.660925 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0702ba4e-e841-41ff-b150-0d30adf8aa49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6cdfe21105c74d4299e7fe2f9b4ac23d)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:15.679112 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0702ba4e-e841-41ff-b150-0d30adf8aa49 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:15.735296 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f66536-eeb3-40f7-8696-652953baf4fc None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:15.740673 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28a6ff6-7cd9-484f-8517-aba3d0fb2ece service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd05409b38ae346458abfe41ebda1c59e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:15.741930 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28a6ff6-7cd9-484f-8517-aba3d0fb2ece service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:15.742266 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28a6ff6-7cd9-484f-8517-aba3d0fb2ece service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:15.743472 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28a6ff6-7cd9-484f-8517-aba3d0fb2ece service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:15.751623 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c28a6ff6-7cd9-484f-8517-aba3d0fb2ece service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:15.769390 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28a6ff6-7cd9-484f-8517-aba3d0fb2ece service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:15.771842 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28a6ff6-7cd9-484f-8517-aba3d0fb2ece service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:15.800999 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 936/1868] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:30:15 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:30:15.822603 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28a6ff6-7cd9-484f-8517-aba3d0fb2ece service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:15.822603 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c28a6ff6-7cd9-484f-8517-aba3d0fb2ece service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:15.891224 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28a6ff6-7cd9-484f-8517-aba3d0fb2ece service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:15.896280 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135fad9a-2bd9-4de9-bd88-391fdc83f911 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:15.897229 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135fad9a-2bd9-4de9-bd88-391fdc83f911 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:15.897615 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135fad9a-2bd9-4de9-bd88-391fdc83f911 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:15.897954 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135fad9a-2bd9-4de9-bd88-391fdc83f911 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:15.900264 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-135fad9a-2bd9-4de9-bd88-391fdc83f911 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:15.914388 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135fad9a-2bd9-4de9-bd88-391fdc83f911 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:15.916158 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135fad9a-2bd9-4de9-bd88-391fdc83f911 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:15.929219 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 937/1869] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:30:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:30:16.062301 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-0702ba4e-e841-41ff-b150-0d30adf8aa49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6cdfe21105c74d4299e7fe2f9b4ac23d'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:16.076765 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-0702ba4e-e841-41ff-b150-0d30adf8aa49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6cdfe21105c74d4299e7fe2f9b4ac23d'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:16.088869 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-0702ba4e-e841-41ff-b150-0d30adf8aa49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6cdfe21105c74d4299e7fe2f9b4ac23d'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:16.090260 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-0702ba4e-e841-41ff-b150-0d30adf8aa49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6cdfe21105c74d4299e7fe2f9b4ac23d'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:16.090707 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-0702ba4e-e841-41ff-b150-0d30adf8aa49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6cdfe21105c74d4299e7fe2f9b4ac23d'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:16.171718 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135fad9a-2bd9-4de9-bd88-391fdc83f911 service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:16.173256 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-135fad9a-2bd9-4de9-bd88-391fdc83f911 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:16.187992 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135fad9a-2bd9-4de9-bd88-391fdc83f911 service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:16.193152 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58915f1b-879e-40bd-9e5d-73a69eb573ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:16.194077 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58915f1b-879e-40bd-9e5d-73a69eb573ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:16.194655 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58915f1b-879e-40bd-9e5d-73a69eb573ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:16.194934 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58915f1b-879e-40bd-9e5d-73a69eb573ed admin admin] PATH_INFO: `/v3/users/8fd2822c3a944a4ba930c76c54e25482` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:16.196224 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58915f1b-879e-40bd-9e5d-73a69eb573ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=8fd2822c3a944a4ba930c76c54e25482)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:16.198142 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58915f1b-879e-40bd-9e5d-73a69eb573ed admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:16.216338 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 933/1870] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 17 19:30:15 2022] DELETE /identity/v3/users/6cdfe21105c74d4299e7fe2f9b4ac23d => generated 0 bytes in 586 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 17 19:30:16.230477 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0702ba4e-e841-41ff-b150-0d30adf8aa49 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:16.232648 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-0702ba4e-e841-41ff-b150-0d30adf8aa49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:16.291387 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0702ba4e-e841-41ff-b150-0d30adf8aa49 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:16.298083 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647bf05e-3e72-4531-b351-5dd070b51975 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd05409b38ae346458abfe41ebda1c59e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:16.306148 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647bf05e-3e72-4531-b351-5dd070b51975 service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:16.306148 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647bf05e-3e72-4531-b351-5dd070b51975 service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:16.306148 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647bf05e-3e72-4531-b351-5dd070b51975 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:16.306148 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-647bf05e-3e72-4531-b351-5dd070b51975 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:16.379267 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647bf05e-3e72-4531-b351-5dd070b51975 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:16.380965 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647bf05e-3e72-4531-b351-5dd070b51975 service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:16.393668 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 934/1871] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:30:16 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 17 19:30:16.422981 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-58915f1b-879e-40bd-9e5d-73a69eb573ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8fd2822c3a944a4ba930c76c54e25482'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:16.426178 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-58915f1b-879e-40bd-9e5d-73a69eb573ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8fd2822c3a944a4ba930c76c54e25482'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:16.459226 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-58915f1b-879e-40bd-9e5d-73a69eb573ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8fd2822c3a944a4ba930c76c54e25482'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:16.548213 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-58915f1b-879e-40bd-9e5d-73a69eb573ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8fd2822c3a944a4ba930c76c54e25482'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:16.555434 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-58915f1b-879e-40bd-9e5d-73a69eb573ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8fd2822c3a944a4ba930c76c54e25482'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:16.562013 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 938/1872] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 17 19:30:16 2022] DELETE /identity/v3/users/8fd2822c3a944a4ba930c76c54e25482 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 17 19:30:16.566545 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d3b692-b13a-45c3-a697-4b038b511a08 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:16.567011 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d3b692-b13a-45c3-a697-4b038b511a08 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:16.567121 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d3b692-b13a-45c3-a697-4b038b511a08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:16.594231 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-15d3b692-b13a-45c3-a697-4b038b511a08 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:30:16.594231 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-15d3b692-b13a-45c3-a697-4b038b511a08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:16.594517 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58915f1b-879e-40bd-9e5d-73a69eb573ed admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:16.594517 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-58915f1b-879e-40bd-9e5d-73a69eb573ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:16.659316 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 935/1873] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:30:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:30:16.668658 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d3b692-b13a-45c3-a697-4b038b511a08 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:16.670338 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-15d3b692-b13a-45c3-a697-4b038b511a08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:16.739730 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58915f1b-879e-40bd-9e5d-73a69eb573ed admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:16.744865 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c02d5f0-459b-41bb-82ef-555988e4d449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:16.745749 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c02d5f0-459b-41bb-82ef-555988e4d449 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:16.746449 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c02d5f0-459b-41bb-82ef-555988e4d449 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:16.746774 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c02d5f0-459b-41bb-82ef-555988e4d449 admin admin] PATH_INFO: `/v3/projects/d3d28e72f0c84bc0960a840bf11d33a2` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:16.748428 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c02d5f0-459b-41bb-82ef-555988e4d449 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3d28e72f0c84bc0960a840bf11d33a2)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:16.750353 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c02d5f0-459b-41bb-82ef-555988e4d449 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:16.760324 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d3b692-b13a-45c3-a697-4b038b511a08 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:16.774440 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9665e304-3eef-4c9e-b7bb-a8b379cd2a40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:16.775105 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9665e304-3eef-4c9e-b7bb-a8b379cd2a40 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:16.775382 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9665e304-3eef-4c9e-b7bb-a8b379cd2a40 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:16.775685 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9665e304-3eef-4c9e-b7bb-a8b379cd2a40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:16.786008 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9665e304-3eef-4c9e-b7bb-a8b379cd2a40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:16.787952 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9665e304-3eef-4c9e-b7bb-a8b379cd2a40 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:16.800004 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 936/1874] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:30:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 17 19:30:16.898322 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-8c02d5f0-459b-41bb-82ef-555988e4d449 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3d28e72f0c84bc0960a840bf11d33a2', 'request_id': 'req-8c02d5f0-459b-41bb-82ef-555988e4d449'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:16.904902 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-8c02d5f0-459b-41bb-82ef-555988e4d449 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3d28e72f0c84bc0960a840bf11d33a2', 'request_id': 'req-8c02d5f0-459b-41bb-82ef-555988e4d449'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:17.008826 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9665e304-3eef-4c9e-b7bb-a8b379cd2a40 service neutron] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:17.010717 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-9665e304-3eef-4c9e-b7bb-a8b379cd2a40 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:17.043390 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9665e304-3eef-4c9e-b7bb-a8b379cd2a40 service neutron] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:17.046252 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-8c02d5f0-459b-41bb-82ef-555988e4d449 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3d28e72f0c84bc0960a840bf11d33a2', 'request_id': 'req-8c02d5f0-459b-41bb-82ef-555988e4d449'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:17.071421 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-8c02d5f0-459b-41bb-82ef-555988e4d449 admin admin] The token cache is being invalidate because project d3d28e72f0c84bc0960a840bf11d33a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 17 19:30:17.071730 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-8c02d5f0-459b-41bb-82ef-555988e4d449 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:17.075966 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-8c02d5f0-459b-41bb-82ef-555988e4d449 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:17.079792 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 939/1875] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 17 19:30:16 2022] DELETE /identity/v3/projects/d3d28e72f0c84bc0960a840bf11d33a2 => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 17 19:30:17.084639 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c02d5f0-459b-41bb-82ef-555988e4d449 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:17.086221 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-8c02d5f0-459b-41bb-82ef-555988e4d449 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:17.200312 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1b7003-7768-40cb-b0f4-2ad41c68add0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:17.200606 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1b7003-7768-40cb-b0f4-2ad41c68add0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:17.200841 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1b7003-7768-40cb-b0f4-2ad41c68add0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:17.201110 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1b7003-7768-40cb-b0f4-2ad41c68add0 admin admin] PATH_INFO: `/v3/projects/20aaad0091b140d688414a4ae8cb61d8` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:17.203112 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1b7003-7768-40cb-b0f4-2ad41c68add0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=20aaad0091b140d688414a4ae8cb61d8)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:17.205227 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1b7003-7768-40cb-b0f4-2ad41c68add0 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:17.281741 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c02d5f0-459b-41bb-82ef-555988e4d449 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:17.287220 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf49e8d8-9fc2-4ce0-b228-b5b990b3d9c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:17.288237 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf49e8d8-9fc2-4ce0-b228-b5b990b3d9c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:17.288587 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf49e8d8-9fc2-4ce0-b228-b5b990b3d9c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:17.288929 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf49e8d8-9fc2-4ce0-b228-b5b990b3d9c6 admin admin] PATH_INFO: `/v3/users/2cf4daf86ed448868c36a8fec2a5ac6b` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:17.293753 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf49e8d8-9fc2-4ce0-b228-b5b990b3d9c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2cf4daf86ed448868c36a8fec2a5ac6b)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:17.297348 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf49e8d8-9fc2-4ce0-b228-b5b990b3d9c6 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:17.379443 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-ef1b7003-7768-40cb-b0f4-2ad41c68add0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20aaad0091b140d688414a4ae8cb61d8', 'request_id': 'req-ef1b7003-7768-40cb-b0f4-2ad41c68add0'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:17.386546 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-ef1b7003-7768-40cb-b0f4-2ad41c68add0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20aaad0091b140d688414a4ae8cb61d8', 'request_id': 'req-ef1b7003-7768-40cb-b0f4-2ad41c68add0'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:17.394301 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-ef1b7003-7768-40cb-b0f4-2ad41c68add0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20aaad0091b140d688414a4ae8cb61d8', 'request_id': 'req-ef1b7003-7768-40cb-b0f4-2ad41c68add0'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:17.570753 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-ef1b7003-7768-40cb-b0f4-2ad41c68add0 admin admin] The token cache is being invalidate because project 20aaad0091b140d688414a4ae8cb61d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 17 19:30:17.571381 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-ef1b7003-7768-40cb-b0f4-2ad41c68add0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:17.571588 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-ef1b7003-7768-40cb-b0f4-2ad41c68add0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:17.573121 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 937/1876] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 17 19:30:17 2022] DELETE /identity/v3/projects/20aaad0091b140d688414a4ae8cb61d8 => generated 0 bytes in 565 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 17 19:30:17.573944 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1b7003-7768-40cb-b0f4-2ad41c68add0 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:17.575151 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-ef1b7003-7768-40cb-b0f4-2ad41c68add0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:17.637453 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1b7003-7768-40cb-b0f4-2ad41c68add0 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:17.644406 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dd4073-e52f-4c35-9176-9fb58f6b044e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:17.644546 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dd4073-e52f-4c35-9176-9fb58f6b044e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:17.644801 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dd4073-e52f-4c35-9176-9fb58f6b044e admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:17.645216 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dd4073-e52f-4c35-9176-9fb58f6b044e admin admin] PATH_INFO: `/v3/users/3bad77b438024885a54e0de1b0d821b1` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:17.646265 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dd4073-e52f-4c35-9176-9fb58f6b044e admin admin] RBAC: Authorizing `identity:delete_user(user_id=3bad77b438024885a54e0de1b0d821b1)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:17.648275 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dd4073-e52f-4c35-9176-9fb58f6b044e admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:17.649343 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-cf49e8d8-9fc2-4ce0-b228-b5b990b3d9c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2cf4daf86ed448868c36a8fec2a5ac6b'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:17.650320 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-cf49e8d8-9fc2-4ce0-b228-b5b990b3d9c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2cf4daf86ed448868c36a8fec2a5ac6b'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:17.666065 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-cf49e8d8-9fc2-4ce0-b228-b5b990b3d9c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2cf4daf86ed448868c36a8fec2a5ac6b'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:17.731363 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-cf49e8d8-9fc2-4ce0-b228-b5b990b3d9c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2cf4daf86ed448868c36a8fec2a5ac6b'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:17.739536 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-cf49e8d8-9fc2-4ce0-b228-b5b990b3d9c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2cf4daf86ed448868c36a8fec2a5ac6b'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:17.745479 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 940/1877] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 17 19:30:17 2022] DELETE /identity/v3/users/2cf4daf86ed448868c36a8fec2a5ac6b => generated 0 bytes in 661 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:30:17.747926 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18e0a68-3368-43a9-8ce4-191436f8fe14 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:17.747926 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18e0a68-3368-43a9-8ce4-191436f8fe14 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:17.747926 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18e0a68-3368-43a9-8ce4-191436f8fe14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:17.766824 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-f18e0a68-3368-43a9-8ce4-191436f8fe14 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:30:17.768056 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-f18e0a68-3368-43a9-8ce4-191436f8fe14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:17.806305 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-73dd4073-e52f-4c35-9176-9fb58f6b044e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3bad77b438024885a54e0de1b0d821b1'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:17.812703 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-73dd4073-e52f-4c35-9176-9fb58f6b044e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3bad77b438024885a54e0de1b0d821b1'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:17.826033 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-73dd4073-e52f-4c35-9176-9fb58f6b044e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3bad77b438024885a54e0de1b0d821b1'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:17.827080 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-73dd4073-e52f-4c35-9176-9fb58f6b044e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3bad77b438024885a54e0de1b0d821b1'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:17.827832 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-73dd4073-e52f-4c35-9176-9fb58f6b044e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3bad77b438024885a54e0de1b0d821b1'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:17.836515 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 941/1878] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:30:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:30:17.842722 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18e0a68-3368-43a9-8ce4-191436f8fe14 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:17.842722 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-f18e0a68-3368-43a9-8ce4-191436f8fe14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:17.851384 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18e0a68-3368-43a9-8ce4-191436f8fe14 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:17.855445 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a6ed03-1a89-4ded-a55d-540b2a788f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:17.856322 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a6ed03-1a89-4ded-a55d-540b2a788f67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:17.857177 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a6ed03-1a89-4ded-a55d-540b2a788f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:17.857450 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a6ed03-1a89-4ded-a55d-540b2a788f67 admin admin] PATH_INFO: `/v3/projects/8cc067a77f4f4cb3927903c895afcb51` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:17.858650 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a6ed03-1a89-4ded-a55d-540b2a788f67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8cc067a77f4f4cb3927903c895afcb51)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:17.860595 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a6ed03-1a89-4ded-a55d-540b2a788f67 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:17.925886 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 938/1879] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 17 19:30:17 2022] DELETE /identity/v3/users/3bad77b438024885a54e0de1b0d821b1 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:30:17.926790 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dd4073-e52f-4c35-9176-9fb58f6b044e admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:17.928170 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-73dd4073-e52f-4c35-9176-9fb58f6b044e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:17.992707 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dd4073-e52f-4c35-9176-9fb58f6b044e admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:17.997993 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1c9694-096e-496c-80eb-b173e73ad697 tempest-VolumeQuotaClassesTest-312823185 tempest-VolumeQuotaClassesTest-312823185-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6300d36694c84482afa1ef40c48b9259', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cbd020abc4747c9aba7f8d0ddad5d02', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:17.998640 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1c9694-096e-496c-80eb-b173e73ad697 tempest-VolumeQuotaClassesTest-312823185 tempest-VolumeQuotaClassesTest-312823185-project-admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:17.999023 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1c9694-096e-496c-80eb-b173e73ad697 tempest-VolumeQuotaClassesTest-312823185 tempest-VolumeQuotaClassesTest-312823185-project-admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:17.999186 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1c9694-096e-496c-80eb-b173e73ad697 tempest-VolumeQuotaClassesTest-312823185 tempest-VolumeQuotaClassesTest-312823185-project-admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:18.000068 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1c9694-096e-496c-80eb-b173e73ad697 tempest-VolumeQuotaClassesTest-312823185 tempest-VolumeQuotaClassesTest-312823185-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:30:18.006652 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1c9694-096e-496c-80eb-b173e73ad697 tempest-VolumeQuotaClassesTest-312823185 tempest-VolumeQuotaClassesTest-312823185-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:18.006652 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1c9694-096e-496c-80eb-b173e73ad697 tempest-VolumeQuotaClassesTest-312823185 tempest-VolumeQuotaClassesTest-312823185-project-admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:18.014746 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 939/1880] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 17 19:30:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:30:18.017128 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55362fd6-0f1e-4e4a-a866-a39596dee41e None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:18.017452 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55362fd6-0f1e-4e4a-a866-a39596dee41e None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:18.018322 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55362fd6-0f1e-4e4a-a866-a39596dee41e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:18.037401 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-55362fd6-0f1e-4e4a-a866-a39596dee41e None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:30:18.038697 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-55362fd6-0f1e-4e4a-a866-a39596dee41e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:18.070475 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d7a6ed03-1a89-4ded-a55d-540b2a788f67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8cc067a77f4f4cb3927903c895afcb51', 'request_id': 'req-d7a6ed03-1a89-4ded-a55d-540b2a788f67'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:18.078310 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d7a6ed03-1a89-4ded-a55d-540b2a788f67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8cc067a77f4f4cb3927903c895afcb51', 'request_id': 'req-d7a6ed03-1a89-4ded-a55d-540b2a788f67'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:18.160106 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d7a6ed03-1a89-4ded-a55d-540b2a788f67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8cc067a77f4f4cb3927903c895afcb51', 'request_id': 'req-d7a6ed03-1a89-4ded-a55d-540b2a788f67'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:18.173615 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d7a6ed03-1a89-4ded-a55d-540b2a788f67 admin admin] The token cache is being invalidate because project 8cc067a77f4f4cb3927903c895afcb51 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 17 19:30:18.174073 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d7a6ed03-1a89-4ded-a55d-540b2a788f67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:18.174778 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d7a6ed03-1a89-4ded-a55d-540b2a788f67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:18.179671 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 942/1881] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 17 19:30:17 2022] DELETE /identity/v3/projects/8cc067a77f4f4cb3927903c895afcb51 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:30:18.179671 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f295fdb1-382a-43ed-b500-d1e003c98e3e None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:18.179671 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f295fdb1-382a-43ed-b500-d1e003c98e3e None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:18.179671 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f295fdb1-382a-43ed-b500-d1e003c98e3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:18.217163 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-f295fdb1-382a-43ed-b500-d1e003c98e3e None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:30:18.218743 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-f295fdb1-382a-43ed-b500-d1e003c98e3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:18.240587 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 940/1882] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:30:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:30:18.243751 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55362fd6-0f1e-4e4a-a866-a39596dee41e None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:18.246390 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-55362fd6-0f1e-4e4a-a866-a39596dee41e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:18.261043 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55362fd6-0f1e-4e4a-a866-a39596dee41e None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:18.299443 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 943/1883] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:30:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:30:18.300414 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f295fdb1-382a-43ed-b500-d1e003c98e3e None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:18.301916 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-f295fdb1-382a-43ed-b500-d1e003c98e3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:18.315508 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f295fdb1-382a-43ed-b500-d1e003c98e3e None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:18.322405 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e68e3a-2394-42ff-b54b-43fbd3d68271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:18.323356 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e68e3a-2394-42ff-b54b-43fbd3d68271 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:18.323707 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e68e3a-2394-42ff-b54b-43fbd3d68271 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:18.323986 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e68e3a-2394-42ff-b54b-43fbd3d68271 admin admin] PATH_INFO: `/v3/users/fa7751acd00b4a468082ad9c77626aa6` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:18.325342 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e68e3a-2394-42ff-b54b-43fbd3d68271 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa7751acd00b4a468082ad9c77626aa6)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:18.329171 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e68e3a-2394-42ff-b54b-43fbd3d68271 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:18.339081 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2027780e-bc01-4c03-84fa-170da4d916f8 tempest-VolumeQuotaClassesTest-312823185 tempest-VolumeQuotaClassesTest-312823185-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6300d36694c84482afa1ef40c48b9259', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cbd020abc4747c9aba7f8d0ddad5d02', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:18.339966 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2027780e-bc01-4c03-84fa-170da4d916f8 tempest-VolumeQuotaClassesTest-312823185 tempest-VolumeQuotaClassesTest-312823185-project-admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:18.340320 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2027780e-bc01-4c03-84fa-170da4d916f8 tempest-VolumeQuotaClassesTest-312823185 tempest-VolumeQuotaClassesTest-312823185-project-admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:18.341314 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2027780e-bc01-4c03-84fa-170da4d916f8 tempest-VolumeQuotaClassesTest-312823185 tempest-VolumeQuotaClassesTest-312823185-project-admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:18.342033 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2027780e-bc01-4c03-84fa-170da4d916f8 tempest-VolumeQuotaClassesTest-312823185 tempest-VolumeQuotaClassesTest-312823185-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:18.344504 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2027780e-bc01-4c03-84fa-170da4d916f8 tempest-VolumeQuotaClassesTest-312823185 tempest-VolumeQuotaClassesTest-312823185-project-admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:18.454431 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 941/1884] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 17 19:30:18 2022] POST /identity/v3/projects => generated 353 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:30:18.454885 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2027780e-bc01-4c03-84fa-170da4d916f8 tempest-VolumeQuotaClassesTest-312823185 tempest-VolumeQuotaClassesTest-312823185-project-admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:18.462384 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-2027780e-bc01-4c03-84fa-170da4d916f8 tempest-VolumeQuotaClassesTest-312823185 tempest-VolumeQuotaClassesTest-312823185-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:18.534895 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2027780e-bc01-4c03-84fa-170da4d916f8 tempest-VolumeQuotaClassesTest-312823185 tempest-VolumeQuotaClassesTest-312823185-project-admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:18.537427 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ee90b6-5ff0-4689-af78-4ebabe6918ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:18.538279 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ee90b6-5ff0-4689-af78-4ebabe6918ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:18.538563 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ee90b6-5ff0-4689-af78-4ebabe6918ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:18.538849 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ee90b6-5ff0-4689-af78-4ebabe6918ff admin admin] PATH_INFO: `/v3/projects/f23d43ffed3d43b2baa79d453c5c2f91` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:18.540108 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ee90b6-5ff0-4689-af78-4ebabe6918ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=f23d43ffed3d43b2baa79d453c5c2f91)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:18.544868 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ee90b6-5ff0-4689-af78-4ebabe6918ff admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:18.585086 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-95e68e3a-2394-42ff-b54b-43fbd3d68271 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa7751acd00b4a468082ad9c77626aa6'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:18.585086 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-95e68e3a-2394-42ff-b54b-43fbd3d68271 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa7751acd00b4a468082ad9c77626aa6'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:18.594801 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-95e68e3a-2394-42ff-b54b-43fbd3d68271 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa7751acd00b4a468082ad9c77626aa6'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:18.650660 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-65ee90b6-5ff0-4689-af78-4ebabe6918ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f23d43ffed3d43b2baa79d453c5c2f91', 'request_id': 'req-65ee90b6-5ff0-4689-af78-4ebabe6918ff'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:18.656738 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-65ee90b6-5ff0-4689-af78-4ebabe6918ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f23d43ffed3d43b2baa79d453c5c2f91', 'request_id': 'req-65ee90b6-5ff0-4689-af78-4ebabe6918ff'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:18.661970 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-65ee90b6-5ff0-4689-af78-4ebabe6918ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f23d43ffed3d43b2baa79d453c5c2f91', 'request_id': 'req-65ee90b6-5ff0-4689-af78-4ebabe6918ff'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:18.666231 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-95e68e3a-2394-42ff-b54b-43fbd3d68271 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa7751acd00b4a468082ad9c77626aa6'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:18.673977 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-95e68e3a-2394-42ff-b54b-43fbd3d68271 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa7751acd00b4a468082ad9c77626aa6'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:18.680635 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 944/1885] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 17 19:30:18 2022] DELETE /identity/v3/users/fa7751acd00b4a468082ad9c77626aa6 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:30:18.681420 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e68e3a-2394-42ff-b54b-43fbd3d68271 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:18.682735 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-95e68e3a-2394-42ff-b54b-43fbd3d68271 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:18.745548 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-65ee90b6-5ff0-4689-af78-4ebabe6918ff admin admin] The token cache is being invalidate because project f23d43ffed3d43b2baa79d453c5c2f91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 17 19:30:18.746188 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-65ee90b6-5ff0-4689-af78-4ebabe6918ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:18.747304 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-65ee90b6-5ff0-4689-af78-4ebabe6918ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:18.752722 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 942/1886] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 17 19:30:18 2022] DELETE /identity/v3/projects/f23d43ffed3d43b2baa79d453c5c2f91 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:30:18.753924 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ee90b6-5ff0-4689-af78-4ebabe6918ff admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:18.756304 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e68e3a-2394-42ff-b54b-43fbd3d68271 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:18.756607 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-65ee90b6-5ff0-4689-af78-4ebabe6918ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:18.768106 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82b99e8-c279-4594-9752-4c8e6d406928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:18.768785 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82b99e8-c279-4594-9752-4c8e6d406928 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:18.772181 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82b99e8-c279-4594-9752-4c8e6d406928 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:18.772181 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82b99e8-c279-4594-9752-4c8e6d406928 admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:18.772181 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82b99e8-c279-4594-9752-4c8e6d406928 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:30:18.772181 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82b99e8-c279-4594-9752-4c8e6d406928 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:18.772975 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82b99e8-c279-4594-9752-4c8e6d406928 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:18.788382 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 945/1887] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 17 19:30:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:30:18.792829 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82b99e8-c279-4594-9752-4c8e6d406928 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:18.795146 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c82b99e8-c279-4594-9752-4c8e6d406928 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:18.838035 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ee90b6-5ff0-4689-af78-4ebabe6918ff admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:18.869633 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82b99e8-c279-4594-9752-4c8e6d406928 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:18.875055 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4648a4f1-0c90-489b-bb84-f9b7bdbaa79f tempest-VolumeQuotaClassesTest-312823185 tempest-VolumeQuotaClassesTest-312823185-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6300d36694c84482afa1ef40c48b9259', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cbd020abc4747c9aba7f8d0ddad5d02', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:18.875811 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4648a4f1-0c90-489b-bb84-f9b7bdbaa79f tempest-VolumeQuotaClassesTest-312823185 tempest-VolumeQuotaClassesTest-312823185-project-admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:18.876112 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4648a4f1-0c90-489b-bb84-f9b7bdbaa79f tempest-VolumeQuotaClassesTest-312823185 tempest-VolumeQuotaClassesTest-312823185-project-admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:18.876332 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4648a4f1-0c90-489b-bb84-f9b7bdbaa79f tempest-VolumeQuotaClassesTest-312823185 tempest-VolumeQuotaClassesTest-312823185-project-admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:18.877104 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4648a4f1-0c90-489b-bb84-f9b7bdbaa79f tempest-VolumeQuotaClassesTest-312823185 tempest-VolumeQuotaClassesTest-312823185-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:30:18.877418 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4648a4f1-0c90-489b-bb84-f9b7bdbaa79f tempest-VolumeQuotaClassesTest-312823185 tempest-VolumeQuotaClassesTest-312823185-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:18.879146 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4648a4f1-0c90-489b-bb84-f9b7bdbaa79f tempest-VolumeQuotaClassesTest-312823185 tempest-VolumeQuotaClassesTest-312823185-project-admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:18.895082 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 946/1888] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 17 19:30:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:30:18.895082 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f461ac7b-1ae6-4f63-814e-bf51f05c4d46 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:18.895082 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f461ac7b-1ae6-4f63-814e-bf51f05c4d46 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:18.895082 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f461ac7b-1ae6-4f63-814e-bf51f05c4d46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:18.904765 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cc3237-6874-4ab7-9f42-a343f79b9f04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:18.904765 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cc3237-6874-4ab7-9f42-a343f79b9f04 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:18.904765 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cc3237-6874-4ab7-9f42-a343f79b9f04 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:18.904765 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cc3237-6874-4ab7-9f42-a343f79b9f04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:18.906808 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-75cc3237-6874-4ab7-9f42-a343f79b9f04 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:18.915707 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-f461ac7b-1ae6-4f63-814e-bf51f05c4d46 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:30:18.916925 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-f461ac7b-1ae6-4f63-814e-bf51f05c4d46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:18.922938 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cc3237-6874-4ab7-9f42-a343f79b9f04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:18.925303 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cc3237-6874-4ab7-9f42-a343f79b9f04 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:18.926225 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 947/1889] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:30:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:30:18.927086 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f461ac7b-1ae6-4f63-814e-bf51f05c4d46 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:18.934603 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-f461ac7b-1ae6-4f63-814e-bf51f05c4d46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:18.941381 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 943/1890] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:30:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:30:18.943283 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21df91df-d28d-407b-b43c-b770ef459541 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:18.943820 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21df91df-d28d-407b-b43c-b770ef459541 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:18.944301 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21df91df-d28d-407b-b43c-b770ef459541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:18.955170 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f461ac7b-1ae6-4f63-814e-bf51f05c4d46 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:18.960331 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eeae25d-05e2-47cb-b8ec-55f2b70554ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:18.964563 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeae25d-05e2-47cb-b8ec-55f2b70554ca admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:18.964563 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeae25d-05e2-47cb-b8ec-55f2b70554ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:18.964563 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeae25d-05e2-47cb-b8ec-55f2b70554ca admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:18.964563 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eeae25d-05e2-47cb-b8ec-55f2b70554ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:18.964563 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eeae25d-05e2-47cb-b8ec-55f2b70554ca admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:18.964936 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-21df91df-d28d-407b-b43c-b770ef459541 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:30:18.966606 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-21df91df-d28d-407b-b43c-b770ef459541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:18.977529 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 944/1891] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:30:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:30:18.979700 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21df91df-d28d-407b-b43c-b770ef459541 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:18.990108 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21df91df-d28d-407b-b43c-b770ef459541 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:18.994891 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bcae0a-1125-4691-b461-f3e0c67c45f1 tempest-VolumeQuotaClassesTest-312823185 tempest-VolumeQuotaClassesTest-312823185-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6300d36694c84482afa1ef40c48b9259', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5cbd020abc4747c9aba7f8d0ddad5d02', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:18.996152 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bcae0a-1125-4691-b461-f3e0c67c45f1 tempest-VolumeQuotaClassesTest-312823185 tempest-VolumeQuotaClassesTest-312823185-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:18.996437 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bcae0a-1125-4691-b461-f3e0c67c45f1 tempest-VolumeQuotaClassesTest-312823185 tempest-VolumeQuotaClassesTest-312823185-project-admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:18.996721 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bcae0a-1125-4691-b461-f3e0c67c45f1 tempest-VolumeQuotaClassesTest-312823185 tempest-VolumeQuotaClassesTest-312823185-project-admin] PATH_INFO: `/v3/projects/bdd2eea4b9b9481db29df430c2e67667` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:18.998040 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bcae0a-1125-4691-b461-f3e0c67c45f1 tempest-VolumeQuotaClassesTest-312823185 tempest-VolumeQuotaClassesTest-312823185-project-admin] RBAC: Authorizing `identity:delete_project(project_id=bdd2eea4b9b9481db29df430c2e67667)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:19.000019 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bcae0a-1125-4691-b461-f3e0c67c45f1 tempest-VolumeQuotaClassesTest-312823185 tempest-VolumeQuotaClassesTest-312823185-project-admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:19.078457 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 948/1892] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 17 19:30:18 2022] POST /identity/v3/projects => generated 387 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:30:19.081143 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eeae25d-05e2-47cb-b8ec-55f2b70554ca admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:19.144606 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-61bcae0a-1125-4691-b461-f3e0c67c45f1 tempest-VolumeQuotaClassesTest-312823185 tempest-VolumeQuotaClassesTest-312823185-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bdd2eea4b9b9481db29df430c2e67667', 'request_id': 'req-61bcae0a-1125-4691-b461-f3e0c67c45f1'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:19.158271 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-61bcae0a-1125-4691-b461-f3e0c67c45f1 tempest-VolumeQuotaClassesTest-312823185 tempest-VolumeQuotaClassesTest-312823185-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bdd2eea4b9b9481db29df430c2e67667', 'request_id': 'req-61bcae0a-1125-4691-b461-f3e0c67c45f1'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:19.166764 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eeae25d-05e2-47cb-b8ec-55f2b70554ca admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:19.168175 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-61bcae0a-1125-4691-b461-f3e0c67c45f1 tempest-VolumeQuotaClassesTest-312823185 tempest-VolumeQuotaClassesTest-312823185-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bdd2eea4b9b9481db29df430c2e67667', 'request_id': 'req-61bcae0a-1125-4691-b461-f3e0c67c45f1'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:19.235745 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72195cb-cdaf-4844-9394-6745681a6b2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:19.236437 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72195cb-cdaf-4844-9394-6745681a6b2b admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:19.236721 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72195cb-cdaf-4844-9394-6745681a6b2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:19.236992 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72195cb-cdaf-4844-9394-6745681a6b2b admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:19.237710 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72195cb-cdaf-4844-9394-6745681a6b2b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:30:19.238396 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72195cb-cdaf-4844-9394-6745681a6b2b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:19.239542 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-61bcae0a-1125-4691-b461-f3e0c67c45f1 tempest-VolumeQuotaClassesTest-312823185 tempest-VolumeQuotaClassesTest-312823185-project-admin] The token cache is being invalidate because project bdd2eea4b9b9481db29df430c2e67667 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 17 19:30:19.241227 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72195cb-cdaf-4844-9394-6745681a6b2b admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:19.241338 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-61bcae0a-1125-4691-b461-f3e0c67c45f1 tempest-VolumeQuotaClassesTest-312823185 tempest-VolumeQuotaClassesTest-312823185-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:19.241338 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-61bcae0a-1125-4691-b461-f3e0c67c45f1 tempest-VolumeQuotaClassesTest-312823185 tempest-VolumeQuotaClassesTest-312823185-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:19.242587 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 945/1893] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 17 19:30:18 2022] DELETE /identity/v3/projects/bdd2eea4b9b9481db29df430c2e67667 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:30:19.243302 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bcae0a-1125-4691-b461-f3e0c67c45f1 tempest-VolumeQuotaClassesTest-312823185 tempest-VolumeQuotaClassesTest-312823185-project-admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:19.244713 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-61bcae0a-1125-4691-b461-f3e0c67c45f1 tempest-VolumeQuotaClassesTest-312823185 tempest-VolumeQuotaClassesTest-312823185-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:19.260031 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 949/1894] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 17 19:30:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:30:19.263571 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72195cb-cdaf-4844-9394-6745681a6b2b admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:19.265808 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d72195cb-cdaf-4844-9394-6745681a6b2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:19.266224 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bcae0a-1125-4691-b461-f3e0c67c45f1 tempest-VolumeQuotaClassesTest-312823185 tempest-VolumeQuotaClassesTest-312823185-project-admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:19.271113 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1020b9f0-82f5-4f86-859d-db7394604ed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:19.271922 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1020b9f0-82f5-4f86-859d-db7394604ed5 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:19.272302 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1020b9f0-82f5-4f86-859d-db7394604ed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:19.272677 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1020b9f0-82f5-4f86-859d-db7394604ed5 admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:19.273462 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1020b9f0-82f5-4f86-859d-db7394604ed5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:19.275634 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1020b9f0-82f5-4f86-859d-db7394604ed5 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:19.281278 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72195cb-cdaf-4844-9394-6745681a6b2b admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:19.286988 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b7cbf2-e18e-493f-863d-a6968ff97e11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:19.287795 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b7cbf2-e18e-493f-863d-a6968ff97e11 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:19.288085 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b7cbf2-e18e-493f-863d-a6968ff97e11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:19.288360 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b7cbf2-e18e-493f-863d-a6968ff97e11 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:19.289052 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b7cbf2-e18e-493f-863d-a6968ff97e11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:19.291880 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b7cbf2-e18e-493f-863d-a6968ff97e11 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:19.376282 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 946/1895] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 17 19:30:19 2022] POST /identity/v3/users => generated 294 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:30:19.377439 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1020b9f0-82f5-4f86-859d-db7394604ed5 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:19.378563 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 950/1896] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 17 19:30:19 2022] POST /identity/v3/projects => generated 385 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:30:19.382287 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b7cbf2-e18e-493f-863d-a6968ff97e11 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:19.385454 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-1020b9f0-82f5-4f86-859d-db7394604ed5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:19.399024 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1020b9f0-82f5-4f86-859d-db7394604ed5 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:19.466358 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b7cbf2-e18e-493f-863d-a6968ff97e11 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:19.469255 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04363670-f7e8-4c89-a6c4-d85f27a60ec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:19.470412 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04363670-f7e8-4c89-a6c4-d85f27a60ec6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:19.470796 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04363670-f7e8-4c89-a6c4-d85f27a60ec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:19.471121 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04363670-f7e8-4c89-a6c4-d85f27a60ec6 admin admin] PATH_INFO: `/v3/projects/fa693b3ca120463a8d990d2406524331` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:19.472617 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bdd249-2ec6-4b28-8096-8436cb05d331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:19.472767 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04363670-f7e8-4c89-a6c4-d85f27a60ec6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa693b3ca120463a8d990d2406524331)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:19.473492 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bdd249-2ec6-4b28-8096-8436cb05d331 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:19.474076 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bdd249-2ec6-4b28-8096-8436cb05d331 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:19.474417 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bdd249-2ec6-4b28-8096-8436cb05d331 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:19.474636 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04363670-f7e8-4c89-a6c4-d85f27a60ec6 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:19.475419 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bdd249-2ec6-4b28-8096-8436cb05d331 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:19.477221 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bdd249-2ec6-4b28-8096-8436cb05d331 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:19.489676 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 951/1897] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:30:19 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:30:19.490634 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bdd249-2ec6-4b28-8096-8436cb05d331 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:19.502118 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bdd249-2ec6-4b28-8096-8436cb05d331 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:19.511794 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3947f1d1-613b-4d77-9bc4-2442ec460341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:19.512598 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3947f1d1-613b-4d77-9bc4-2442ec460341 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:19.513153 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3947f1d1-613b-4d77-9bc4-2442ec460341 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:19.513494 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3947f1d1-613b-4d77-9bc4-2442ec460341 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:19.514260 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3947f1d1-613b-4d77-9bc4-2442ec460341 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:19.516363 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3947f1d1-613b-4d77-9bc4-2442ec460341 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:19.631679 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-04363670-f7e8-4c89-a6c4-d85f27a60ec6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa693b3ca120463a8d990d2406524331', 'request_id': 'req-04363670-f7e8-4c89-a6c4-d85f27a60ec6'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:19.645524 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 952/1898] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 17 19:30:19 2022] POST /identity/v3/users => generated 293 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:30:19.646441 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3947f1d1-613b-4d77-9bc4-2442ec460341 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:19.657346 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-04363670-f7e8-4c89-a6c4-d85f27a60ec6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa693b3ca120463a8d990d2406524331', 'request_id': 'req-04363670-f7e8-4c89-a6c4-d85f27a60ec6'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:19.663803 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3947f1d1-613b-4d77-9bc4-2442ec460341 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:19.668304 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-04363670-f7e8-4c89-a6c4-d85f27a60ec6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa693b3ca120463a8d990d2406524331', 'request_id': 'req-04363670-f7e8-4c89-a6c4-d85f27a60ec6'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:19.731737 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651a7de2-574c-4b2e-98d0-4c5d283390de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:19.732558 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651a7de2-574c-4b2e-98d0-4c5d283390de admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:19.732847 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651a7de2-574c-4b2e-98d0-4c5d283390de admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:19.733119 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651a7de2-574c-4b2e-98d0-4c5d283390de admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:19.733842 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651a7de2-574c-4b2e-98d0-4c5d283390de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:19.735519 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651a7de2-574c-4b2e-98d0-4c5d283390de admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:19.747729 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 953/1899] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:30:19 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:30:19.748706 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651a7de2-574c-4b2e-98d0-4c5d283390de admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:19.764492 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651a7de2-574c-4b2e-98d0-4c5d283390de admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:19.782939 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-04363670-f7e8-4c89-a6c4-d85f27a60ec6 admin admin] The token cache is being invalidate because project fa693b3ca120463a8d990d2406524331 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 17 19:30:19.784291 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-04363670-f7e8-4c89-a6c4-d85f27a60ec6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:19.787010 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-04363670-f7e8-4c89-a6c4-d85f27a60ec6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:19.787852 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4558889c-4681-45fe-a56d-fc4a57e5d99a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:19.788537 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4558889c-4681-45fe-a56d-fc4a57e5d99a admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:19.788949 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4558889c-4681-45fe-a56d-fc4a57e5d99a admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:19.789227 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4558889c-4681-45fe-a56d-fc4a57e5d99a admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:19.789953 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4558889c-4681-45fe-a56d-fc4a57e5d99a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:19.790241 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 947/1900] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 17 19:30:19 2022] DELETE /identity/v3/projects/fa693b3ca120463a8d990d2406524331 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:30:19.791112 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04363670-f7e8-4c89-a6c4-d85f27a60ec6 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:19.792577 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-04363670-f7e8-4c89-a6c4-d85f27a60ec6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:19.793243 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4558889c-4681-45fe-a56d-fc4a57e5d99a admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:19.808351 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04363670-f7e8-4c89-a6c4-d85f27a60ec6 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:19.813625 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 954/1901] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:30:19 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:30:19.814759 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4558889c-4681-45fe-a56d-fc4a57e5d99a admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:19.816207 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2057c5-7df4-4d88-b138-c13373f8c20f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:19.816582 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-4558889c-4681-45fe-a56d-fc4a57e5d99a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:19.818496 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2057c5-7df4-4d88-b138-c13373f8c20f admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:19.818988 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2057c5-7df4-4d88-b138-c13373f8c20f admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:19.819477 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2057c5-7df4-4d88-b138-c13373f8c20f admin admin] PATH_INFO: `/v3/projects/8dcf3d06de29437ebbd50e15eec83166/users/914ae901011045fb81d83e02cceaea60/roles/114e6a16e63447d1924707ace5aace25` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:19.832353 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4558889c-4681-45fe-a56d-fc4a57e5d99a admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:19.839058 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c683882-d5b7-4d57-a716-54c7b0765bc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:19.840103 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c683882-d5b7-4d57-a716-54c7b0765bc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:19.840381 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c683882-d5b7-4d57-a716-54c7b0765bc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:19.840651 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c683882-d5b7-4d57-a716-54c7b0765bc9 admin admin] PATH_INFO: `/v3/users/43256ad6f4da4d20bc3063a135eaf8d4` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:19.841918 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c683882-d5b7-4d57-a716-54c7b0765bc9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=43256ad6f4da4d20bc3063a135eaf8d4)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:19.843966 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c683882-d5b7-4d57-a716-54c7b0765bc9 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:19.844562 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2057c5-7df4-4d88-b138-c13373f8c20f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8dcf3d06de29437ebbd50e15eec83166, user_id=914ae901011045fb81d83e02cceaea60, role_id=114e6a16e63447d1924707ace5aace25)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:19.850803 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2057c5-7df4-4d88-b138-c13373f8c20f admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:19.921082 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 948/1902] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:30:19 2022] PUT /identity/v3/projects/8dcf3d06de29437ebbd50e15eec83166/users/914ae901011045fb81d83e02cceaea60/roles/114e6a16e63447d1924707ace5aace25 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:30:19.922278 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2057c5-7df4-4d88-b138-c13373f8c20f admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:19.926034 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-ea2057c5-7df4-4d88-b138-c13373f8c20f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:20.021139 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2057c5-7df4-4d88-b138-c13373f8c20f admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:20.027146 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c20f0d6-0976-4570-a278-3aae37c81fef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:20.027778 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c20f0d6-0976-4570-a278-3aae37c81fef admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:20.028087 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c20f0d6-0976-4570-a278-3aae37c81fef admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:20.029114 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c20f0d6-0976-4570-a278-3aae37c81fef admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:20.029839 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c20f0d6-0976-4570-a278-3aae37c81fef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:20.031635 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c20f0d6-0976-4570-a278-3aae37c81fef admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:20.044998 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 949/1903] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:30:19 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:30:20.044998 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c20f0d6-0976-4570-a278-3aae37c81fef admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:20.045235 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-2c20f0d6-0976-4570-a278-3aae37c81fef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:20.058848 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c20f0d6-0976-4570-a278-3aae37c81fef admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:20.063702 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e402ad8f-3419-4b4b-9c85-39efdb181dd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:20.064600 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e402ad8f-3419-4b4b-9c85-39efdb181dd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:20.064872 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e402ad8f-3419-4b4b-9c85-39efdb181dd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:20.065303 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e402ad8f-3419-4b4b-9c85-39efdb181dd4 admin admin] PATH_INFO: `/v3/projects/28807197c3334ba5b24736fa2b50f9f3` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:20.066475 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e402ad8f-3419-4b4b-9c85-39efdb181dd4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=28807197c3334ba5b24736fa2b50f9f3)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:20.068566 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e402ad8f-3419-4b4b-9c85-39efdb181dd4 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:20.165053 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-3c683882-d5b7-4d57-a716-54c7b0765bc9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43256ad6f4da4d20bc3063a135eaf8d4'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:20.165813 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-3c683882-d5b7-4d57-a716-54c7b0765bc9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43256ad6f4da4d20bc3063a135eaf8d4'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:20.177599 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-3c683882-d5b7-4d57-a716-54c7b0765bc9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43256ad6f4da4d20bc3063a135eaf8d4'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:20.228511 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-e402ad8f-3419-4b4b-9c85-39efdb181dd4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28807197c3334ba5b24736fa2b50f9f3', 'request_id': 'req-e402ad8f-3419-4b4b-9c85-39efdb181dd4'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:20.234597 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-e402ad8f-3419-4b4b-9c85-39efdb181dd4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28807197c3334ba5b24736fa2b50f9f3', 'request_id': 'req-e402ad8f-3419-4b4b-9c85-39efdb181dd4'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:20.240214 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-e402ad8f-3419-4b4b-9c85-39efdb181dd4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28807197c3334ba5b24736fa2b50f9f3', 'request_id': 'req-e402ad8f-3419-4b4b-9c85-39efdb181dd4'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:20.314083 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-3c683882-d5b7-4d57-a716-54c7b0765bc9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43256ad6f4da4d20bc3063a135eaf8d4'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:20.321589 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-3c683882-d5b7-4d57-a716-54c7b0765bc9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43256ad6f4da4d20bc3063a135eaf8d4'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:20.328001 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 955/1904] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 17 19:30:19 2022] DELETE /identity/v3/users/43256ad6f4da4d20bc3063a135eaf8d4 => generated 0 bytes in 514 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:30:20.329254 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02226c44-a801-46c0-a3ec-9f3191a25670 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:20.329532 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02226c44-a801-46c0-a3ec-9f3191a25670 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:20.329800 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02226c44-a801-46c0-a3ec-9f3191a25670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:20.357070 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-02226c44-a801-46c0-a3ec-9f3191a25670 None None] MFA Rules not processed for user `914ae901011045fb81d83e02cceaea60`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:30:20.358193 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-02226c44-a801-46c0-a3ec-9f3191a25670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:20.430864 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 956/1905] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:30:20 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:30:20.431736 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02226c44-a801-46c0-a3ec-9f3191a25670 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:20.432978 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-02226c44-a801-46c0-a3ec-9f3191a25670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:20.440515 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-e402ad8f-3419-4b4b-9c85-39efdb181dd4 admin admin] The token cache is being invalidate because project 28807197c3334ba5b24736fa2b50f9f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 17 19:30:20.440860 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-e402ad8f-3419-4b4b-9c85-39efdb181dd4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:20.441551 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-e402ad8f-3419-4b4b-9c85-39efdb181dd4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:20.446010 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 950/1906] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 17 19:30:20 2022] DELETE /identity/v3/projects/28807197c3334ba5b24736fa2b50f9f3 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:30:20.446010 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e402ad8f-3419-4b4b-9c85-39efdb181dd4 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:20.446010 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-e402ad8f-3419-4b4b-9c85-39efdb181dd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:20.495391 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02226c44-a801-46c0-a3ec-9f3191a25670 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:20.500966 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321aa6a4-061c-4c87-b2c1-5daffb7289c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:20.507211 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321aa6a4-061c-4c87-b2c1-5daffb7289c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:20.507211 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321aa6a4-061c-4c87-b2c1-5daffb7289c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:20.507211 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321aa6a4-061c-4c87-b2c1-5daffb7289c0 admin admin] PATH_INFO: `/v3/projects/fd0d8bc91c7c426596a3e706f5ed7856/users/b516f9768e66489d8c555fcb4c803e10/roles/114e6a16e63447d1924707ace5aace25` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:20.510432 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e402ad8f-3419-4b4b-9c85-39efdb181dd4 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:20.515282 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3927f3-e952-49c8-9a2d-e049eb10b913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:20.516202 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3927f3-e952-49c8-9a2d-e049eb10b913 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:20.516482 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3927f3-e952-49c8-9a2d-e049eb10b913 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:20.516752 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3927f3-e952-49c8-9a2d-e049eb10b913 admin admin] PATH_INFO: `/v3/users/84877a41a7ee477dac6164f0efe7d65d` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:20.517626 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321aa6a4-061c-4c87-b2c1-5daffb7289c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd0d8bc91c7c426596a3e706f5ed7856, user_id=b516f9768e66489d8c555fcb4c803e10, role_id=114e6a16e63447d1924707ace5aace25)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:20.517922 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3927f3-e952-49c8-9a2d-e049eb10b913 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84877a41a7ee477dac6164f0efe7d65d)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:20.519869 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3927f3-e952-49c8-9a2d-e049eb10b913 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:20.520350 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321aa6a4-061c-4c87-b2c1-5daffb7289c0 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:20.631871 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 957/1907] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:30:20 2022] PUT /identity/v3/projects/fd0d8bc91c7c426596a3e706f5ed7856/users/b516f9768e66489d8c555fcb4c803e10/roles/114e6a16e63447d1924707ace5aace25 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:30:20.642355 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321aa6a4-061c-4c87-b2c1-5daffb7289c0 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:20.652338 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-321aa6a4-061c-4c87-b2c1-5daffb7289c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:20.726078 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-5c3927f3-e952-49c8-9a2d-e049eb10b913 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84877a41a7ee477dac6164f0efe7d65d'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:20.735521 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-5c3927f3-e952-49c8-9a2d-e049eb10b913 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84877a41a7ee477dac6164f0efe7d65d'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:20.741490 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321aa6a4-061c-4c87-b2c1-5daffb7289c0 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:20.747683 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f91c0b-6f75-49b2-934f-1b9e7f7c51ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:20.748344 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f91c0b-6f75-49b2-934f-1b9e7f7c51ba admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:20.748618 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f91c0b-6f75-49b2-934f-1b9e7f7c51ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:20.748885 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f91c0b-6f75-49b2-934f-1b9e7f7c51ba admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:20.749650 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f91c0b-6f75-49b2-934f-1b9e7f7c51ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:20.749766 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-5c3927f3-e952-49c8-9a2d-e049eb10b913 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84877a41a7ee477dac6164f0efe7d65d'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:20.750473 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-5c3927f3-e952-49c8-9a2d-e049eb10b913 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84877a41a7ee477dac6164f0efe7d65d'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:20.751133 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-5c3927f3-e952-49c8-9a2d-e049eb10b913 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84877a41a7ee477dac6164f0efe7d65d'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:20.752014 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f91c0b-6f75-49b2-934f-1b9e7f7c51ba admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:20.868854 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 951/1908] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 17 19:30:20 2022] DELETE /identity/v3/users/84877a41a7ee477dac6164f0efe7d65d => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:30:20.873340 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3927f3-e952-49c8-9a2d-e049eb10b913 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:20.878577 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-5c3927f3-e952-49c8-9a2d-e049eb10b913 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:20.916173 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 958/1909] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 17 19:30:20 2022] POST /identity/v3/projects => generated 383 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:30:20.917472 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a101d8-6e5d-4914-befe-34804b8f28ce None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:20.918175 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a101d8-6e5d-4914-befe-34804b8f28ce None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:20.918175 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a101d8-6e5d-4914-befe-34804b8f28ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:20.948841 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3927f3-e952-49c8-9a2d-e049eb10b913 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:20.950923 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-a5a101d8-6e5d-4914-befe-34804b8f28ce None None] MFA Rules not processed for user `b516f9768e66489d8c555fcb4c803e10`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:30:20.952416 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a5a101d8-6e5d-4914-befe-34804b8f28ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:21.011310 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1335048f-a240-428f-bf7f-727b3999877a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:21.012213 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1335048f-a240-428f-bf7f-727b3999877a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:21.012498 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1335048f-a240-428f-bf7f-727b3999877a admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:21.012876 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1335048f-a240-428f-bf7f-727b3999877a admin admin] PATH_INFO: `/v3/users/d4c8804ee59340cc84367cdbe9f21967` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:21.014089 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1335048f-a240-428f-bf7f-727b3999877a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4c8804ee59340cc84367cdbe9f21967)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:21.016173 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1335048f-a240-428f-bf7f-727b3999877a admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:21.018139 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 959/1910] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:30:20 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:30:21.019069 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a101d8-6e5d-4914-befe-34804b8f28ce None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:21.020732 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a5a101d8-6e5d-4914-befe-34804b8f28ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:21.035123 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a101d8-6e5d-4914-befe-34804b8f28ce None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:21.045833 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf7ec54-2ddf-469c-8dcc-84c6b35f2d66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:21.046695 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf7ec54-2ddf-469c-8dcc-84c6b35f2d66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:21.046969 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf7ec54-2ddf-469c-8dcc-84c6b35f2d66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:21.047231 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf7ec54-2ddf-469c-8dcc-84c6b35f2d66 admin admin] PATH_INFO: `/v3/projects/4884f06209034b2c85462aab113ad3b9` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:21.048490 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf7ec54-2ddf-469c-8dcc-84c6b35f2d66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4884f06209034b2c85462aab113ad3b9)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:21.050391 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf7ec54-2ddf-469c-8dcc-84c6b35f2d66 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:21.289568 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-1335048f-a240-428f-bf7f-727b3999877a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4c8804ee59340cc84367cdbe9f21967'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:21.289783 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-fdf7ec54-2ddf-469c-8dcc-84c6b35f2d66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4884f06209034b2c85462aab113ad3b9', 'request_id': 'req-fdf7ec54-2ddf-469c-8dcc-84c6b35f2d66'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:21.296170 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-fdf7ec54-2ddf-469c-8dcc-84c6b35f2d66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4884f06209034b2c85462aab113ad3b9', 'request_id': 'req-fdf7ec54-2ddf-469c-8dcc-84c6b35f2d66'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:21.301676 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-1335048f-a240-428f-bf7f-727b3999877a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4c8804ee59340cc84367cdbe9f21967'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:21.314728 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-1335048f-a240-428f-bf7f-727b3999877a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4c8804ee59340cc84367cdbe9f21967'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:21.315460 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-1335048f-a240-428f-bf7f-727b3999877a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4c8804ee59340cc84367cdbe9f21967'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:21.316199 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-1335048f-a240-428f-bf7f-727b3999877a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4c8804ee59340cc84367cdbe9f21967'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:21.371262 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-fdf7ec54-2ddf-469c-8dcc-84c6b35f2d66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4884f06209034b2c85462aab113ad3b9', 'request_id': 'req-fdf7ec54-2ddf-469c-8dcc-84c6b35f2d66'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:21.382678 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-fdf7ec54-2ddf-469c-8dcc-84c6b35f2d66 admin admin] The token cache is being invalidate because project 4884f06209034b2c85462aab113ad3b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 17 19:30:21.382899 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-fdf7ec54-2ddf-469c-8dcc-84c6b35f2d66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:21.383563 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-fdf7ec54-2ddf-469c-8dcc-84c6b35f2d66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:21.385108 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 960/1911] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 17 19:30:21 2022] DELETE /identity/v3/projects/4884f06209034b2c85462aab113ad3b9 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:30:21.386412 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4002f76-771d-4d07-b259-bbdce6843157 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:21.386698 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4002f76-771d-4d07-b259-bbdce6843157 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:21.386972 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4002f76-771d-4d07-b259-bbdce6843157 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:21.404942 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-e4002f76-771d-4d07-b259-bbdce6843157 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:30:21.406133 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-e4002f76-771d-4d07-b259-bbdce6843157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:21.468574 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 961/1912] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:30:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:30:21.468574 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4002f76-771d-4d07-b259-bbdce6843157 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:21.471699 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-e4002f76-771d-4d07-b259-bbdce6843157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:21.483969 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4002f76-771d-4d07-b259-bbdce6843157 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:21.489068 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54b5adf-cff6-4014-a8d7-beb386518a2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:21.489802 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54b5adf-cff6-4014-a8d7-beb386518a2d admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:21.490154 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54b5adf-cff6-4014-a8d7-beb386518a2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:21.490496 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54b5adf-cff6-4014-a8d7-beb386518a2d admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:21.491248 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54b5adf-cff6-4014-a8d7-beb386518a2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:21.492585 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 952/1913] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 17 19:30:20 2022] DELETE /identity/v3/users/d4c8804ee59340cc84367cdbe9f21967 => generated 0 bytes in 623 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:30:21.493451 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54b5adf-cff6-4014-a8d7-beb386518a2d admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:21.494026 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1335048f-a240-428f-bf7f-727b3999877a admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:21.497581 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-1335048f-a240-428f-bf7f-727b3999877a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:21.581605 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1335048f-a240-428f-bf7f-727b3999877a admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:21.586517 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a48243-7c52-42c7-85b1-4340bfd31cea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:21.587181 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a48243-7c52-42c7-85b1-4340bfd31cea admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:21.587466 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a48243-7c52-42c7-85b1-4340bfd31cea admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:21.587784 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a48243-7c52-42c7-85b1-4340bfd31cea admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:21.588476 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a48243-7c52-42c7-85b1-4340bfd31cea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:21.590488 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a48243-7c52-42c7-85b1-4340bfd31cea admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:21.606297 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 962/1914] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 17 19:30:21 2022] POST /identity/v3/users => generated 298 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:30:21.607966 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817f9c0b-87a9-4068-929a-7ca54f35a1a3 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:21.608257 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817f9c0b-87a9-4068-929a-7ca54f35a1a3 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:21.608537 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817f9c0b-87a9-4068-929a-7ca54f35a1a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:21.632261 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-817f9c0b-87a9-4068-929a-7ca54f35a1a3 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:30:21.635451 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-817f9c0b-87a9-4068-929a-7ca54f35a1a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:21.642999 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 963/1915] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:30:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:30:21.644478 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817f9c0b-87a9-4068-929a-7ca54f35a1a3 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:21.645703 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-817f9c0b-87a9-4068-929a-7ca54f35a1a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:21.705310 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 953/1916] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 17 19:30:21 2022] POST /identity/v3/projects => generated 385 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:30:21.709186 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a48243-7c52-42c7-85b1-4340bfd31cea admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:21.713671 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817f9c0b-87a9-4068-929a-7ca54f35a1a3 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:21.770789 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee0124c-55a5-481b-ad0d-78b2541f662f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:21.771059 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee0124c-55a5-481b-ad0d-78b2541f662f service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:21.771193 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee0124c-55a5-481b-ad0d-78b2541f662f service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:21.771472 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee0124c-55a5-481b-ad0d-78b2541f662f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:21.854239 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a48243-7c52-42c7-85b1-4340bfd31cea admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:21.863215 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee0124c-55a5-481b-ad0d-78b2541f662f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:21.864926 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee0124c-55a5-481b-ad0d-78b2541f662f service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:21.867169 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b5a78d-2d0c-494a-a609-7837c0fd8277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:21.873090 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b5a78d-2d0c-494a-a609-7837c0fd8277 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:21.873090 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b5a78d-2d0c-494a-a609-7837c0fd8277 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:21.873090 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b5a78d-2d0c-494a-a609-7837c0fd8277 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:21.873090 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b5a78d-2d0c-494a-a609-7837c0fd8277 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:21.873090 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b5a78d-2d0c-494a-a609-7837c0fd8277 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:21.876348 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 964/1917] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:30:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:30:21.877840 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30f5573-effe-4838-a25b-d5fbd0e68a76 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:21.878206 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30f5573-effe-4838-a25b-d5fbd0e68a76 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:21.878489 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30f5573-effe-4838-a25b-d5fbd0e68a76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:21.888775 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 954/1918] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:30:21 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:30:21.889752 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b5a78d-2d0c-494a-a609-7837c0fd8277 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:21.891799 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-f4b5a78d-2d0c-494a-a609-7837c0fd8277 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:21.903544 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-e30f5573-effe-4838-a25b-d5fbd0e68a76 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:30:21.904757 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-e30f5573-effe-4838-a25b-d5fbd0e68a76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:21.910066 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b5a78d-2d0c-494a-a609-7837c0fd8277 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:21.916350 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f196df58-0dae-44a2-9029-9568ad61044b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:21.917138 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f196df58-0dae-44a2-9029-9568ad61044b admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:21.917669 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f196df58-0dae-44a2-9029-9568ad61044b admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:21.918098 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f196df58-0dae-44a2-9029-9568ad61044b admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:21.919004 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f196df58-0dae-44a2-9029-9568ad61044b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:21.921258 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f196df58-0dae-44a2-9029-9568ad61044b admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:21.936327 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 965/1919] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:30:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:30:21.937522 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30f5573-effe-4838-a25b-d5fbd0e68a76 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:21.948097 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30f5573-effe-4838-a25b-d5fbd0e68a76 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:21.954487 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a828d2a-e2e8-4ad1-9b63-17c88ce0fb60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:21.955387 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a828d2a-e2e8-4ad1-9b63-17c88ce0fb60 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:21.955845 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a828d2a-e2e8-4ad1-9b63-17c88ce0fb60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:21.956040 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a828d2a-e2e8-4ad1-9b63-17c88ce0fb60 admin admin] PATH_INFO: `/v3/projects/cc489143a7284b8fb1b691c2f666a361/users/13c35e4239514a60b42c16b1e216db87/roles/60af5088793e4379b88821a86552a344` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:21.972730 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a828d2a-e2e8-4ad1-9b63-17c88ce0fb60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc489143a7284b8fb1b691c2f666a361, user_id=13c35e4239514a60b42c16b1e216db87, role_id=60af5088793e4379b88821a86552a344)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:21.976086 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a828d2a-e2e8-4ad1-9b63-17c88ce0fb60 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:22.022607 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 955/1920] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 17 19:30:21 2022] POST /identity/v3/users => generated 299 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:30:22.023588 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f196df58-0dae-44a2-9029-9568ad61044b admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:22.034600 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f196df58-0dae-44a2-9029-9568ad61044b admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:22.040629 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff84803-6d10-4af2-bdd1-a6b8f9a49a25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:22.041809 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff84803-6d10-4af2-bdd1-a6b8f9a49a25 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:22.042101 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff84803-6d10-4af2-bdd1-a6b8f9a49a25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:22.042381 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff84803-6d10-4af2-bdd1-a6b8f9a49a25 admin admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:22.043270 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff84803-6d10-4af2-bdd1-a6b8f9a49a25 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:30:22.043634 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff84803-6d10-4af2-bdd1-a6b8f9a49a25 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:22.046329 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff84803-6d10-4af2-bdd1-a6b8f9a49a25 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:22.072230 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 956/1921] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 17 19:30:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:30:22.072934 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff84803-6d10-4af2-bdd1-a6b8f9a49a25 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:22.086444 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff84803-6d10-4af2-bdd1-a6b8f9a49a25 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:22.093128 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 966/1922] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:30:21 2022] PUT /identity/v3/projects/cc489143a7284b8fb1b691c2f666a361/users/13c35e4239514a60b42c16b1e216db87/roles/60af5088793e4379b88821a86552a344 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:30:22.093128 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a828d2a-e2e8-4ad1-9b63-17c88ce0fb60 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:22.184378 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f56e8f-a977-4484-8ec7-112e85e830df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:22.185705 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f56e8f-a977-4484-8ec7-112e85e830df admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:22.186091 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f56e8f-a977-4484-8ec7-112e85e830df admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:22.186920 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f56e8f-a977-4484-8ec7-112e85e830df admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:22.187986 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f56e8f-a977-4484-8ec7-112e85e830df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:22.190352 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f56e8f-a977-4484-8ec7-112e85e830df admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:22.203243 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 957/1923] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:30:22 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:30:22.204302 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f56e8f-a977-4484-8ec7-112e85e830df admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:22.214608 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f56e8f-a977-4484-8ec7-112e85e830df admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:22.222320 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaa27ae-1e80-4acf-b64c-63a228d52a3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:22.223358 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaa27ae-1e80-4acf-b64c-63a228d52a3d admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:22.225273 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaa27ae-1e80-4acf-b64c-63a228d52a3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:22.225273 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaa27ae-1e80-4acf-b64c-63a228d52a3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:22.225273 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaa27ae-1e80-4acf-b64c-63a228d52a3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:22.225557 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a828d2a-e2e8-4ad1-9b63-17c88ce0fb60 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:22.226988 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaa27ae-1e80-4acf-b64c-63a228d52a3d admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:22.230624 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44078ea-ce35-4170-9c0e-1508b95ab0d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:22.231949 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44078ea-ce35-4170-9c0e-1508b95ab0d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:22.232298 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44078ea-ce35-4170-9c0e-1508b95ab0d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:22.232638 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44078ea-ce35-4170-9c0e-1508b95ab0d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:22.233510 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44078ea-ce35-4170-9c0e-1508b95ab0d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:22.235576 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44078ea-ce35-4170-9c0e-1508b95ab0d6 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:22.240257 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 958/1924] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:30:22 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:30:22.240801 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaa27ae-1e80-4acf-b64c-63a228d52a3d admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:22.252896 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaa27ae-1e80-4acf-b64c-63a228d52a3d admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:22.258118 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6966b64-1fb8-4ebb-8f19-bd1c75ab994a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:22.259029 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6966b64-1fb8-4ebb-8f19-bd1c75ab994a admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:22.259306 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6966b64-1fb8-4ebb-8f19-bd1c75ab994a admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:22.262501 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6966b64-1fb8-4ebb-8f19-bd1c75ab994a admin admin] PATH_INFO: `/v3/projects/a246b8459f3243019886b94e9f4288d7/users/07d1e046c9be49d7a889f4461ba35ca5/roles/60af5088793e4379b88821a86552a344` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:22.274435 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6966b64-1fb8-4ebb-8f19-bd1c75ab994a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a246b8459f3243019886b94e9f4288d7, user_id=07d1e046c9be49d7a889f4461ba35ca5, role_id=60af5088793e4379b88821a86552a344)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:22.277175 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6966b64-1fb8-4ebb-8f19-bd1c75ab994a admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:22.377018 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 967/1925] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 17 19:30:22 2022] POST /identity/v3/projects => generated 377 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:30:22.378134 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44078ea-ce35-4170-9c0e-1508b95ab0d6 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:22.415288 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 959/1926] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:30:22 2022] PUT /identity/v3/projects/a246b8459f3243019886b94e9f4288d7/users/07d1e046c9be49d7a889f4461ba35ca5/roles/60af5088793e4379b88821a86552a344 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:30:22.417222 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6966b64-1fb8-4ebb-8f19-bd1c75ab994a admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:22.419392 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-b6966b64-1fb8-4ebb-8f19-bd1c75ab994a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:22.449392 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44078ea-ce35-4170-9c0e-1508b95ab0d6 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:22.493294 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6966b64-1fb8-4ebb-8f19-bd1c75ab994a admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:22.499213 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8902e5-b9a0-43de-a3ea-744e8c1ed3dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:22.500277 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8902e5-b9a0-43de-a3ea-744e8c1ed3dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:22.500701 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8902e5-b9a0-43de-a3ea-744e8c1ed3dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:22.501114 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8902e5-b9a0-43de-a3ea-744e8c1ed3dc admin admin] PATH_INFO: `/v3/projects/ba1c1e27ce5c437f8496cf330abf9030` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:22.502475 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8902e5-b9a0-43de-a3ea-744e8c1ed3dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba1c1e27ce5c437f8496cf330abf9030)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:22.504619 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8902e5-b9a0-43de-a3ea-744e8c1ed3dc admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:22.517142 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e61ba9d-f300-4ee8-b3f4-5953d8578dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:22.518780 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e61ba9d-f300-4ee8-b3f4-5953d8578dcb admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:22.519233 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e61ba9d-f300-4ee8-b3f4-5953d8578dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:22.519741 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e61ba9d-f300-4ee8-b3f4-5953d8578dcb admin admin] PATH_INFO: `/v3/projects/cc489143a7284b8fb1b691c2f666a361/users/13c35e4239514a60b42c16b1e216db87/roles/60af5088793e4379b88821a86552a344` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:22.522016 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e61ba9d-f300-4ee8-b3f4-5953d8578dcb admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc489143a7284b8fb1b691c2f666a361, user_id=13c35e4239514a60b42c16b1e216db87, role_id=60af5088793e4379b88821a86552a344)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:22.524819 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e61ba9d-f300-4ee8-b3f4-5953d8578dcb admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:22.535961 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 968/1927] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:30:22 2022] PUT /identity/v3/projects/cc489143a7284b8fb1b691c2f666a361/users/13c35e4239514a60b42c16b1e216db87/roles/60af5088793e4379b88821a86552a344 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:30:22.536967 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e61ba9d-f300-4ee8-b3f4-5953d8578dcb admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:22.607812 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e61ba9d-f300-4ee8-b3f4-5953d8578dcb admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:22.612658 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d48f33-0c15-4254-9b0b-e4a741a2a7d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:22.613326 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d48f33-0c15-4254-9b0b-e4a741a2a7d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:22.613615 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d48f33-0c15-4254-9b0b-e4a741a2a7d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:22.613888 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d48f33-0c15-4254-9b0b-e4a741a2a7d8 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:22.614600 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d48f33-0c15-4254-9b0b-e4a741a2a7d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:22.616620 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d48f33-0c15-4254-9b0b-e4a741a2a7d8 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:22.661150 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-ba8902e5-b9a0-43de-a3ea-744e8c1ed3dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba1c1e27ce5c437f8496cf330abf9030', 'request_id': 'req-ba8902e5-b9a0-43de-a3ea-744e8c1ed3dc'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:22.668036 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-ba8902e5-b9a0-43de-a3ea-744e8c1ed3dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba1c1e27ce5c437f8496cf330abf9030', 'request_id': 'req-ba8902e5-b9a0-43de-a3ea-744e8c1ed3dc'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:22.673576 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-ba8902e5-b9a0-43de-a3ea-744e8c1ed3dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba1c1e27ce5c437f8496cf330abf9030', 'request_id': 'req-ba8902e5-b9a0-43de-a3ea-744e8c1ed3dc'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:22.702876 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 969/1928] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 17 19:30:22 2022] POST /identity/v3/users => generated 289 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:30:22.703726 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d48f33-0c15-4254-9b0b-e4a741a2a7d8 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:22.713268 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d48f33-0c15-4254-9b0b-e4a741a2a7d8 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:22.769823 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3d234f-a533-4413-bcee-5fb5a775b769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:22.770484 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3d234f-a533-4413-bcee-5fb5a775b769 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:22.770761 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3d234f-a533-4413-bcee-5fb5a775b769 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:22.771032 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3d234f-a533-4413-bcee-5fb5a775b769 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:22.771774 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3d234f-a533-4413-bcee-5fb5a775b769 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:22.773371 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3d234f-a533-4413-bcee-5fb5a775b769 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:22.779800 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-ba8902e5-b9a0-43de-a3ea-744e8c1ed3dc admin admin] The token cache is being invalidate because project ba1c1e27ce5c437f8496cf330abf9030 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 17 19:30:22.780112 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-ba8902e5-b9a0-43de-a3ea-744e8c1ed3dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:22.782552 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-ba8902e5-b9a0-43de-a3ea-744e8c1ed3dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:22.782552 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 960/1929] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 17 19:30:22 2022] DELETE /identity/v3/projects/ba1c1e27ce5c437f8496cf330abf9030 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:30:22.786525 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 970/1930] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:30:22 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:30:22.786663 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9107e20e-b78e-43a3-afa6-c4cc5abc70b0 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:22.786774 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3d234f-a533-4413-bcee-5fb5a775b769 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:22.786871 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9107e20e-b78e-43a3-afa6-c4cc5abc70b0 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:22.786871 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9107e20e-b78e-43a3-afa6-c4cc5abc70b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:22.787821 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-0b3d234f-a533-4413-bcee-5fb5a775b769 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:22.799895 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3d234f-a533-4413-bcee-5fb5a775b769 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:22.804875 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c196dd13-af27-4b4b-a082-3a14e97fa615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:22.814296 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c196dd13-af27-4b4b-a082-3a14e97fa615 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:22.814296 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c196dd13-af27-4b4b-a082-3a14e97fa615 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:22.814296 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c196dd13-af27-4b4b-a082-3a14e97fa615 admin admin] PATH_INFO: `/v3/projects/71337bc709d54773810da46d85f52d34` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:22.814296 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c196dd13-af27-4b4b-a082-3a14e97fa615 admin admin] RBAC: Authorizing `identity:delete_project(project_id=71337bc709d54773810da46d85f52d34)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:22.814296 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c196dd13-af27-4b4b-a082-3a14e97fa615 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:22.814703 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-9107e20e-b78e-43a3-afa6-c4cc5abc70b0 None None] MFA Rules not processed for user `13c35e4239514a60b42c16b1e216db87`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:30:22.814703 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-9107e20e-b78e-43a3-afa6-c4cc5abc70b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:22.879442 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 961/1931] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:30:22 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:30:22.880462 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9107e20e-b78e-43a3-afa6-c4cc5abc70b0 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:22.883613 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-9107e20e-b78e-43a3-afa6-c4cc5abc70b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:22.910272 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9107e20e-b78e-43a3-afa6-c4cc5abc70b0 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:22.910272 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60fcf2d-5e85-45ec-b112-1d60496569ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:22.910272 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60fcf2d-5e85-45ec-b112-1d60496569ef admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:22.910272 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60fcf2d-5e85-45ec-b112-1d60496569ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:22.911059 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60fcf2d-5e85-45ec-b112-1d60496569ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:22.911059 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60fcf2d-5e85-45ec-b112-1d60496569ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:22.911059 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60fcf2d-5e85-45ec-b112-1d60496569ef admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:22.924760 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 962/1932] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:30:22 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:30:22.926141 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1febaed5-e901-4ea8-9b91-30addb1d8da4 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:22.926423 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1febaed5-e901-4ea8-9b91-30addb1d8da4 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:22.926824 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1febaed5-e901-4ea8-9b91-30addb1d8da4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:22.954399 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-1febaed5-e901-4ea8-9b91-30addb1d8da4 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:30:22.955901 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-1febaed5-e901-4ea8-9b91-30addb1d8da4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:22.957645 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c196dd13-af27-4b4b-a082-3a14e97fa615 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71337bc709d54773810da46d85f52d34', 'request_id': 'req-c196dd13-af27-4b4b-a082-3a14e97fa615'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:22.963347 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c196dd13-af27-4b4b-a082-3a14e97fa615 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71337bc709d54773810da46d85f52d34', 'request_id': 'req-c196dd13-af27-4b4b-a082-3a14e97fa615'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:23.024511 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 963/1933] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:30:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:30:23.025386 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1febaed5-e901-4ea8-9b91-30addb1d8da4 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:23.027174 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-1febaed5-e901-4ea8-9b91-30addb1d8da4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:23.039408 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1febaed5-e901-4ea8-9b91-30addb1d8da4 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:23.044551 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c935c2-fd92-4d18-a838-d13a764b801b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:23.045438 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c935c2-fd92-4d18-a838-d13a764b801b admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:23.045714 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c935c2-fd92-4d18-a838-d13a764b801b admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:23.045987 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c935c2-fd92-4d18-a838-d13a764b801b admin admin] PATH_INFO: `/v3/projects/a246b8459f3243019886b94e9f4288d7/users/07d1e046c9be49d7a889f4461ba35ca5/roles/60af5088793e4379b88821a86552a344` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:23.048292 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c935c2-fd92-4d18-a838-d13a764b801b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a246b8459f3243019886b94e9f4288d7, user_id=07d1e046c9be49d7a889f4461ba35ca5, role_id=60af5088793e4379b88821a86552a344)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:23.051564 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c935c2-fd92-4d18-a838-d13a764b801b admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:23.063177 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 964/1934] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:30:23 2022] PUT /identity/v3/projects/a246b8459f3243019886b94e9f4288d7/users/07d1e046c9be49d7a889f4461ba35ca5/roles/60af5088793e4379b88821a86552a344 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:30:23.067769 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c935c2-fd92-4d18-a838-d13a764b801b admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:23.067769 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-44c935c2-fd92-4d18-a838-d13a764b801b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:23.119246 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c196dd13-af27-4b4b-a082-3a14e97fa615 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71337bc709d54773810da46d85f52d34', 'request_id': 'req-c196dd13-af27-4b4b-a082-3a14e97fa615'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:23.124973 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c935c2-fd92-4d18-a838-d13a764b801b admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:23.128651 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760c7b39-de18-4133-a04f-bc42dc14bd8e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd05409b38ae346458abfe41ebda1c59e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:23.129288 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760c7b39-de18-4133-a04f-bc42dc14bd8e service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:23.129568 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760c7b39-de18-4133-a04f-bc42dc14bd8e service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:23.129840 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760c7b39-de18-4133-a04f-bc42dc14bd8e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:23.133049 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c196dd13-af27-4b4b-a082-3a14e97fa615 admin admin] The token cache is being invalidate because project 71337bc709d54773810da46d85f52d34 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 17 19:30:23.133317 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c196dd13-af27-4b4b-a082-3a14e97fa615 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:23.133950 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c196dd13-af27-4b4b-a082-3a14e97fa615 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:23.135535 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 971/1935] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 17 19:30:22 2022] DELETE /identity/v3/projects/71337bc709d54773810da46d85f52d34 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:30:23.136317 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c196dd13-af27-4b4b-a082-3a14e97fa615 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:23.137623 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c196dd13-af27-4b4b-a082-3a14e97fa615 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:23.149827 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c196dd13-af27-4b4b-a082-3a14e97fa615 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:23.189881 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760c7b39-de18-4133-a04f-bc42dc14bd8e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:23.191249 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760c7b39-de18-4133-a04f-bc42dc14bd8e service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:23.201159 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 965/1936] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:30:23 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:30:23.202421 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10688396-d670-4a63-b2e8-105d5e8f9e9d None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:23.202763 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10688396-d670-4a63-b2e8-105d5e8f9e9d None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:23.202972 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10688396-d670-4a63-b2e8-105d5e8f9e9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:23.209206 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f209ffa5-6592-4bba-9865-e9a06fb1b9ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:23.209878 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f209ffa5-6592-4bba-9865-e9a06fb1b9ac admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:23.210165 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f209ffa5-6592-4bba-9865-e9a06fb1b9ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:23.210441 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f209ffa5-6592-4bba-9865-e9a06fb1b9ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:23.211155 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f209ffa5-6592-4bba-9865-e9a06fb1b9ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:23.212971 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f209ffa5-6592-4bba-9865-e9a06fb1b9ac admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:23.220443 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-10688396-d670-4a63-b2e8-105d5e8f9e9d None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:30:23.221908 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-10688396-d670-4a63-b2e8-105d5e8f9e9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:23.227942 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 972/1937] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:30:23 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:30:23.227942 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fba38ae-a0ed-47bc-9fe2-902e8f09cd58 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:23.227942 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fba38ae-a0ed-47bc-9fe2-902e8f09cd58 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:23.227942 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fba38ae-a0ed-47bc-9fe2-902e8f09cd58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:23.235635 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 966/1938] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:30:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:30:23.235635 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10688396-d670-4a63-b2e8-105d5e8f9e9d None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:23.235635 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-10688396-d670-4a63-b2e8-105d5e8f9e9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:23.246529 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10688396-d670-4a63-b2e8-105d5e8f9e9d None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:23.258599 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed11ea9b-68a4-4017-a3cd-c11040a53ccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:23.259478 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed11ea9b-68a4-4017-a3cd-c11040a53ccf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:23.259786 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed11ea9b-68a4-4017-a3cd-c11040a53ccf admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:23.260100 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed11ea9b-68a4-4017-a3cd-c11040a53ccf admin admin] PATH_INFO: `/v3/users/6300d36694c84482afa1ef40c48b9259` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:23.261349 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed11ea9b-68a4-4017-a3cd-c11040a53ccf admin admin] RBAC: Authorizing `identity:delete_user(user_id=6300d36694c84482afa1ef40c48b9259)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:23.261854 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-1fba38ae-a0ed-47bc-9fe2-902e8f09cd58 None None] MFA Rules not processed for user `07d1e046c9be49d7a889f4461ba35ca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:30:23.263303 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed11ea9b-68a4-4017-a3cd-c11040a53ccf admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:23.263444 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-1fba38ae-a0ed-47bc-9fe2-902e8f09cd58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:23.395054 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 973/1939] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:30:23 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:30:23.396290 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fba38ae-a0ed-47bc-9fe2-902e8f09cd58 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:23.410342 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fba38ae-a0ed-47bc-9fe2-902e8f09cd58 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:23.415241 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d5e3fd-b27a-4eed-bd29-c9a84ffbfdff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:23.416180 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d5e3fd-b27a-4eed-bd29-c9a84ffbfdff admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:23.416452 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d5e3fd-b27a-4eed-bd29-c9a84ffbfdff admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:23.416725 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d5e3fd-b27a-4eed-bd29-c9a84ffbfdff admin admin] PATH_INFO: `/v3/projects/97d6875a858449e7b8062238704a4769/users/df22e0d8a20444debd00e3cde8fc98ce/roles/114e6a16e63447d1924707ace5aace25` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:23.440590 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d5e3fd-b27a-4eed-bd29-c9a84ffbfdff admin admin] RBAC: Authorizing `identity:create_grant(project_id=97d6875a858449e7b8062238704a4769, user_id=df22e0d8a20444debd00e3cde8fc98ce, role_id=114e6a16e63447d1924707ace5aace25)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:23.443669 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d5e3fd-b27a-4eed-bd29-c9a84ffbfdff admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:23.506979 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-ed11ea9b-68a4-4017-a3cd-c11040a53ccf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6300d36694c84482afa1ef40c48b9259'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:23.515505 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-ed11ea9b-68a4-4017-a3cd-c11040a53ccf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6300d36694c84482afa1ef40c48b9259'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:23.522306 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 974/1940] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:30:23 2022] PUT /identity/v3/projects/97d6875a858449e7b8062238704a4769/users/df22e0d8a20444debd00e3cde8fc98ce/roles/114e6a16e63447d1924707ace5aace25 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:30:23.528705 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-ed11ea9b-68a4-4017-a3cd-c11040a53ccf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6300d36694c84482afa1ef40c48b9259'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:23.529355 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d5e3fd-b27a-4eed-bd29-c9a84ffbfdff admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:23.529868 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-ed11ea9b-68a4-4017-a3cd-c11040a53ccf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6300d36694c84482afa1ef40c48b9259'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:23.530719 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-ed11ea9b-68a4-4017-a3cd-c11040a53ccf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6300d36694c84482afa1ef40c48b9259'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:23.589504 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d5e3fd-b27a-4eed-bd29-c9a84ffbfdff admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:23.594218 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de9d066-12fc-4c6a-a1b1-75710ede0649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:23.594867 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de9d066-12fc-4c6a-a1b1-75710ede0649 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:23.595144 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de9d066-12fc-4c6a-a1b1-75710ede0649 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:23.595415 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de9d066-12fc-4c6a-a1b1-75710ede0649 admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:23.596432 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de9d066-12fc-4c6a-a1b1-75710ede0649 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:30:23.596731 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de9d066-12fc-4c6a-a1b1-75710ede0649 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:23.598329 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de9d066-12fc-4c6a-a1b1-75710ede0649 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:23.609324 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 967/1941] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 17 19:30:23 2022] DELETE /identity/v3/users/6300d36694c84482afa1ef40c48b9259 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:30:23.613152 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 975/1942] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 17 19:30:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:30:23.613731 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed11ea9b-68a4-4017-a3cd-c11040a53ccf admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:23.613822 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd01f3c2-fbbc-43b4-b050-0495856df606 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:23.613822 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd01f3c2-fbbc-43b4-b050-0495856df606 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:23.613822 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd01f3c2-fbbc-43b4-b050-0495856df606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:23.614831 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-ed11ea9b-68a4-4017-a3cd-c11040a53ccf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:23.650990 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-fd01f3c2-fbbc-43b4-b050-0495856df606 None None] MFA Rules not processed for user `df22e0d8a20444debd00e3cde8fc98ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:30:23.652724 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-fd01f3c2-fbbc-43b4-b050-0495856df606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:23.689771 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed11ea9b-68a4-4017-a3cd-c11040a53ccf admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:23.693779 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c0317b-6db8-4b48-97f5-9a1bc2664c08 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd05409b38ae346458abfe41ebda1c59e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:23.694408 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c0317b-6db8-4b48-97f5-9a1bc2664c08 service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:23.694696 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c0317b-6db8-4b48-97f5-9a1bc2664c08 service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:23.694959 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c0317b-6db8-4b48-97f5-9a1bc2664c08 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:23.696922 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-39c0317b-6db8-4b48-97f5-9a1bc2664c08 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:23.744592 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 976/1943] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:30:23 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:30:23.745496 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd01f3c2-fbbc-43b4-b050-0495856df606 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:23.746735 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-fd01f3c2-fbbc-43b4-b050-0495856df606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:23.758296 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c0317b-6db8-4b48-97f5-9a1bc2664c08 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:23.759921 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c0317b-6db8-4b48-97f5-9a1bc2664c08 service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:23.769903 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 968/1944] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:30:23 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:30:23.770575 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c0317b-6db8-4b48-97f5-9a1bc2664c08 service cinder] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:23.772627 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-39c0317b-6db8-4b48-97f5-9a1bc2664c08 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:23.814515 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd01f3c2-fbbc-43b4-b050-0495856df606 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:23.833272 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402d724a-892a-491b-8886-d19f8ed907c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:23.836230 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402d724a-892a-491b-8886-d19f8ed907c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:23.836516 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402d724a-892a-491b-8886-d19f8ed907c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:23.836801 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402d724a-892a-491b-8886-d19f8ed907c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:23.844115 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402d724a-892a-491b-8886-d19f8ed907c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:23.846083 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402d724a-892a-491b-8886-d19f8ed907c5 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:23.887214 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c0317b-6db8-4b48-97f5-9a1bc2664c08 service cinder] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:23.896480 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb596a87-7527-43cb-83ca-a696b1349695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:23.896480 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb596a87-7527-43cb-83ca-a696b1349695 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:23.896480 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb596a87-7527-43cb-83ca-a696b1349695 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:23.896480 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb596a87-7527-43cb-83ca-a696b1349695 admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:23.896984 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb596a87-7527-43cb-83ca-a696b1349695 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:23.901393 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb596a87-7527-43cb-83ca-a696b1349695 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:23.966667 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 977/1945] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 17 19:30:23 2022] POST /identity/v3/projects => generated 381 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:30:23.967789 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402d724a-892a-491b-8886-d19f8ed907c5 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:23.989863 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 969/1946] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 17 19:30:23 2022] POST /identity/v3/projects => generated 379 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:30:23.991064 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb596a87-7527-43cb-83ca-a696b1349695 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:24.036379 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402d724a-892a-491b-8886-d19f8ed907c5 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:24.065439 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb596a87-7527-43cb-83ca-a696b1349695 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:24.070341 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1e97ab-6203-498c-a93b-5d982d7e79fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:24.071016 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1e97ab-6203-498c-a93b-5d982d7e79fa admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:24.071297 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1e97ab-6203-498c-a93b-5d982d7e79fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:24.071627 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1e97ab-6203-498c-a93b-5d982d7e79fa admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:24.072670 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1e97ab-6203-498c-a93b-5d982d7e79fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:24.074603 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1e97ab-6203-498c-a93b-5d982d7e79fa admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:24.095096 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4099a6b1-cb90-4a23-983b-9ec1465e15dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:24.095817 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4099a6b1-cb90-4a23-983b-9ec1465e15dc admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:24.096095 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4099a6b1-cb90-4a23-983b-9ec1465e15dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:24.096625 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4099a6b1-cb90-4a23-983b-9ec1465e15dc admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:24.097306 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4099a6b1-cb90-4a23-983b-9ec1465e15dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:24.099273 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4099a6b1-cb90-4a23-983b-9ec1465e15dc admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:24.172362 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 970/1947] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 17 19:30:23 2022] POST /identity/v3/users => generated 290 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:30:24.176040 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1e97ab-6203-498c-a93b-5d982d7e79fa admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:24.177730 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-cb1e97ab-6203-498c-a93b-5d982d7e79fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:24.189905 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1e97ab-6203-498c-a93b-5d982d7e79fa admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:24.198372 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd43a703-ad4a-4c4c-8cc6-797f30fb7dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:24.198372 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd43a703-ad4a-4c4c-8cc6-797f30fb7dd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:24.198372 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd43a703-ad4a-4c4c-8cc6-797f30fb7dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:24.198372 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd43a703-ad4a-4c4c-8cc6-797f30fb7dd5 admin admin] PATH_INFO: `/v3/projects/5cbd020abc4747c9aba7f8d0ddad5d02` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:24.198788 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 978/1948] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 17 19:30:23 2022] POST /identity/v3/users => generated 291 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:30:24.199341 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4099a6b1-cb90-4a23-983b-9ec1465e15dc admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:24.200774 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd43a703-ad4a-4c4c-8cc6-797f30fb7dd5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5cbd020abc4747c9aba7f8d0ddad5d02)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:24.202741 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd43a703-ad4a-4c4c-8cc6-797f30fb7dd5 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:24.209474 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4099a6b1-cb90-4a23-983b-9ec1465e15dc admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:24.216058 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dd34d8-2764-4b7a-8f1b-1c20ede2e9bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:24.216880 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dd34d8-2764-4b7a-8f1b-1c20ede2e9bf admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:24.217225 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dd34d8-2764-4b7a-8f1b-1c20ede2e9bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:24.217672 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dd34d8-2764-4b7a-8f1b-1c20ede2e9bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:24.219061 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dd34d8-2764-4b7a-8f1b-1c20ede2e9bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:24.220883 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dd34d8-2764-4b7a-8f1b-1c20ede2e9bf admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:24.235710 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 979/1949] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:30:24 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:30:24.236703 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dd34d8-2764-4b7a-8f1b-1c20ede2e9bf admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:24.246410 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dd34d8-2764-4b7a-8f1b-1c20ede2e9bf admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:24.251276 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26bb88c-ce55-456b-8079-705dd3fd3462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:24.251986 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26bb88c-ce55-456b-8079-705dd3fd3462 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:24.256608 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26bb88c-ce55-456b-8079-705dd3fd3462 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:24.256608 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26bb88c-ce55-456b-8079-705dd3fd3462 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:24.256608 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26bb88c-ce55-456b-8079-705dd3fd3462 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:24.256608 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26bb88c-ce55-456b-8079-705dd3fd3462 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:24.265590 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 980/1950] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:30:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:30:24.266556 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26bb88c-ce55-456b-8079-705dd3fd3462 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:24.276347 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26bb88c-ce55-456b-8079-705dd3fd3462 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:24.281954 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3104f24-a691-4a15-a28c-eab1c146cc86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:24.282594 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3104f24-a691-4a15-a28c-eab1c146cc86 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:24.282867 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3104f24-a691-4a15-a28c-eab1c146cc86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:24.283135 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3104f24-a691-4a15-a28c-eab1c146cc86 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:24.283932 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3104f24-a691-4a15-a28c-eab1c146cc86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:24.285787 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3104f24-a691-4a15-a28c-eab1c146cc86 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:24.296351 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 981/1951] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:30:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:30:24.297170 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3104f24-a691-4a15-a28c-eab1c146cc86 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:24.306393 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3104f24-a691-4a15-a28c-eab1c146cc86 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:24.311007 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526d7144-c9d5-4551-823e-3a4eed535692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:24.311696 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526d7144-c9d5-4551-823e-3a4eed535692 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:24.311995 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526d7144-c9d5-4551-823e-3a4eed535692 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:24.312245 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526d7144-c9d5-4551-823e-3a4eed535692 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:24.324516 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526d7144-c9d5-4551-823e-3a4eed535692 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:24.324516 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526d7144-c9d5-4551-823e-3a4eed535692 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:24.324516 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 982/1952] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:30:24 2022] GET /identity/v3/roles => generated 1420 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:30:24.324759 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526d7144-c9d5-4551-823e-3a4eed535692 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:24.334252 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526d7144-c9d5-4551-823e-3a4eed535692 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:24.338785 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cef7ab-22ce-450d-a615-4919b21ae682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:24.339625 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cef7ab-22ce-450d-a615-4919b21ae682 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:24.339903 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cef7ab-22ce-450d-a615-4919b21ae682 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:24.340170 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cef7ab-22ce-450d-a615-4919b21ae682 admin admin] PATH_INFO: `/v3/projects/646656a4b9a343e490fa2a954f614ff8/users/095dbdc7dbc942a28bde981ce73684af/roles/114e6a16e63447d1924707ace5aace25` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:24.354974 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cef7ab-22ce-450d-a615-4919b21ae682 admin admin] RBAC: Authorizing `identity:create_grant(project_id=646656a4b9a343e490fa2a954f614ff8, user_id=095dbdc7dbc942a28bde981ce73684af, role_id=114e6a16e63447d1924707ace5aace25)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:24.357734 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cef7ab-22ce-450d-a615-4919b21ae682 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:24.360188 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-fd43a703-ad4a-4c4c-8cc6-797f30fb7dd5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5cbd020abc4747c9aba7f8d0ddad5d02', 'request_id': 'req-fd43a703-ad4a-4c4c-8cc6-797f30fb7dd5'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:24.367181 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-fd43a703-ad4a-4c4c-8cc6-797f30fb7dd5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5cbd020abc4747c9aba7f8d0ddad5d02', 'request_id': 'req-fd43a703-ad4a-4c4c-8cc6-797f30fb7dd5'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:24.372939 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-fd43a703-ad4a-4c4c-8cc6-797f30fb7dd5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5cbd020abc4747c9aba7f8d0ddad5d02', 'request_id': 'req-fd43a703-ad4a-4c4c-8cc6-797f30fb7dd5'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:24.481304 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 983/1953] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:30:24 2022] PUT /identity/v3/projects/646656a4b9a343e490fa2a954f614ff8/users/095dbdc7dbc942a28bde981ce73684af/roles/114e6a16e63447d1924707ace5aace25 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:30:24.490294 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cef7ab-22ce-450d-a615-4919b21ae682 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:24.490397 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-fd43a703-ad4a-4c4c-8cc6-797f30fb7dd5 admin admin] The token cache is being invalidate because project 5cbd020abc4747c9aba7f8d0ddad5d02 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 17 19:30:24.490397 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-fd43a703-ad4a-4c4c-8cc6-797f30fb7dd5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:24.490397 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-fd43a703-ad4a-4c4c-8cc6-797f30fb7dd5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:24.490397 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 971/1954] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 17 19:30:24 2022] DELETE /identity/v3/projects/5cbd020abc4747c9aba7f8d0ddad5d02 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:30:24.492654 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cced04-800f-49ab-af0a-df5b6400dee0 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:24.493243 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cced04-800f-49ab-af0a-df5b6400dee0 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:24.493664 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cced04-800f-49ab-af0a-df5b6400dee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:24.519909 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-43cced04-800f-49ab-af0a-df5b6400dee0 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:30:24.521098 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-43cced04-800f-49ab-af0a-df5b6400dee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:24.574476 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cef7ab-22ce-450d-a615-4919b21ae682 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:24.581269 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4520be-fa58-4a15-9ef6-076a172f9efe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:24.583222 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4520be-fa58-4a15-9ef6-076a172f9efe admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:24.583222 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4520be-fa58-4a15-9ef6-076a172f9efe admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:24.583222 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4520be-fa58-4a15-9ef6-076a172f9efe admin admin] PATH_INFO: `/v3/projects/bfe4515ffcc14ccc81e01855fc4a5309/users/6a537695d7c241808534934d76c58deb/roles/114e6a16e63447d1924707ace5aace25` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:24.604602 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4520be-fa58-4a15-9ef6-076a172f9efe admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfe4515ffcc14ccc81e01855fc4a5309, user_id=6a537695d7c241808534934d76c58deb, role_id=114e6a16e63447d1924707ace5aace25)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:24.609002 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4520be-fa58-4a15-9ef6-076a172f9efe admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:24.614445 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 972/1955] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:30:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 17 19:30:24.615931 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e16d7bc-2eda-42b5-b0b7-4f4f482c121b None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:24.616218 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e16d7bc-2eda-42b5-b0b7-4f4f482c121b None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:24.617608 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e16d7bc-2eda-42b5-b0b7-4f4f482c121b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:24.654003 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-6e16d7bc-2eda-42b5-b0b7-4f4f482c121b None None] MFA Rules not processed for user `095dbdc7dbc942a28bde981ce73684af`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:30:24.655227 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-6e16d7bc-2eda-42b5-b0b7-4f4f482c121b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:24.717255 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 984/1956] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:30:24 2022] PUT /identity/v3/projects/bfe4515ffcc14ccc81e01855fc4a5309/users/6a537695d7c241808534934d76c58deb/roles/114e6a16e63447d1924707ace5aace25 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:30:24.718938 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b75f3a-450c-4c1f-a8f3-d3b4bb8ad995 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:24.719231 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b75f3a-450c-4c1f-a8f3-d3b4bb8ad995 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:24.719529 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b75f3a-450c-4c1f-a8f3-d3b4bb8ad995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:24.742155 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-e2b75f3a-450c-4c1f-a8f3-d3b4bb8ad995 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:30:24.743612 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-e2b75f3a-450c-4c1f-a8f3-d3b4bb8ad995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:24.759451 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 973/1957] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:30:24 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:30:24.761281 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27114aa6-8d58-48b4-90f3-251cf14bf36f None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:24.761712 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27114aa6-8d58-48b4-90f3-251cf14bf36f None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:24.762020 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27114aa6-8d58-48b4-90f3-251cf14bf36f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:24.790647 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-27114aa6-8d58-48b4-90f3-251cf14bf36f None None] MFA Rules not processed for user `6a537695d7c241808534934d76c58deb`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:30:24.791974 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-27114aa6-8d58-48b4-90f3-251cf14bf36f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:24.812799 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 985/1958] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:30:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:30:24.813962 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b75f3a-450c-4c1f-a8f3-d3b4bb8ad995 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:24.815220 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-e2b75f3a-450c-4c1f-a8f3-d3b4bb8ad995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:24.833247 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b75f3a-450c-4c1f-a8f3-d3b4bb8ad995 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:24.838220 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2be1cd-1cb5-44c6-bdc5-7dbaddc8fed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:24.842374 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2be1cd-1cb5-44c6-bdc5-7dbaddc8fed9 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:24.842374 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2be1cd-1cb5-44c6-bdc5-7dbaddc8fed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:24.842374 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2be1cd-1cb5-44c6-bdc5-7dbaddc8fed9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:24.842374 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2be1cd-1cb5-44c6-bdc5-7dbaddc8fed9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:24.842374 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2be1cd-1cb5-44c6-bdc5-7dbaddc8fed9 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:24.891296 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 974/1959] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:30:24 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:30:24.892436 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27114aa6-8d58-48b4-90f3-251cf14bf36f None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:24.906060 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27114aa6-8d58-48b4-90f3-251cf14bf36f None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:24.911840 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df35c80-3a22-4ab3-b4c8-af987391d7ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:24.912587 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df35c80-3a22-4ab3-b4c8-af987391d7ca admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:24.912945 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df35c80-3a22-4ab3-b4c8-af987391d7ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:24.913376 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df35c80-3a22-4ab3-b4c8-af987391d7ca admin admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:24.914330 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df35c80-3a22-4ab3-b4c8-af987391d7ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:30:24.914711 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df35c80-3a22-4ab3-b4c8-af987391d7ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:24.916497 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df35c80-3a22-4ab3-b4c8-af987391d7ca admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:24.930267 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 986/1960] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 17 19:30:24 2022] POST /identity/v3/projects => generated 379 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:30:24.931412 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2be1cd-1cb5-44c6-bdc5-7dbaddc8fed9 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:24.933442 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-3d2be1cd-1cb5-44c6-bdc5-7dbaddc8fed9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:24.935628 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 975/1961] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 17 19:30:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:30:24.936701 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df35c80-3a22-4ab3-b4c8-af987391d7ca admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:24.951354 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df35c80-3a22-4ab3-b4c8-af987391d7ca admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:25.012876 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2be1cd-1cb5-44c6-bdc5-7dbaddc8fed9 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:25.017622 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd75254-fb29-465f-b61d-a6d5e30f1539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:25.018277 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd75254-fb29-465f-b61d-a6d5e30f1539 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:25.018549 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd75254-fb29-465f-b61d-a6d5e30f1539 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:25.018818 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd75254-fb29-465f-b61d-a6d5e30f1539 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:25.019504 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd75254-fb29-465f-b61d-a6d5e30f1539 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:25.021106 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9793a7d5-16a1-49f2-ad4a-96516481055f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:25.021525 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd75254-fb29-465f-b61d-a6d5e30f1539 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:25.022228 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9793a7d5-16a1-49f2-ad4a-96516481055f admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:25.022618 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9793a7d5-16a1-49f2-ad4a-96516481055f admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:25.022899 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9793a7d5-16a1-49f2-ad4a-96516481055f admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:25.023657 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9793a7d5-16a1-49f2-ad4a-96516481055f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:25.025575 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9793a7d5-16a1-49f2-ad4a-96516481055f admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:25.153049 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 987/1962] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 17 19:30:24 2022] POST /identity/v3/projects => generated 381 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:30:25.153049 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd75254-fb29-465f-b61d-a6d5e30f1539 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:25.240483 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd75254-fb29-465f-b61d-a6d5e30f1539 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:25.249091 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff3b212-5c24-426c-ab80-90bf69726444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:25.249874 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff3b212-5c24-426c-ab80-90bf69726444 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:25.258503 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff3b212-5c24-426c-ab80-90bf69726444 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:25.258859 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff3b212-5c24-426c-ab80-90bf69726444 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:25.259613 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff3b212-5c24-426c-ab80-90bf69726444 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:25.261589 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff3b212-5c24-426c-ab80-90bf69726444 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:25.301205 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 976/1963] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 17 19:30:24 2022] POST /identity/v3/users => generated 296 bytes in 362 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:30:25.301205 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9793a7d5-16a1-49f2-ad4a-96516481055f admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:25.311076 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9793a7d5-16a1-49f2-ad4a-96516481055f admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:25.315653 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5de728-f5a0-41ad-9b30-5865c8bac871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:25.316387 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5de728-f5a0-41ad-9b30-5865c8bac871 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:25.316733 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5de728-f5a0-41ad-9b30-5865c8bac871 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:25.317073 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5de728-f5a0-41ad-9b30-5865c8bac871 admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:25.317863 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5de728-f5a0-41ad-9b30-5865c8bac871 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:25.319795 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5de728-f5a0-41ad-9b30-5865c8bac871 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:25.418233 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 988/1964] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 17 19:30:25 2022] POST /identity/v3/projects => generated 383 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:30:25.419187 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff3b212-5c24-426c-ab80-90bf69726444 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:25.420887 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-7ff3b212-5c24-426c-ab80-90bf69726444 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:25.456917 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 977/1965] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 17 19:30:25 2022] POST /identity/v3/users => generated 297 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:30:25.459088 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5de728-f5a0-41ad-9b30-5865c8bac871 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:25.473527 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5de728-f5a0-41ad-9b30-5865c8bac871 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:25.488767 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff3b212-5c24-426c-ab80-90bf69726444 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:25.493814 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e86082d-6359-4b73-aeb4-75e93f468b9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:25.494719 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e86082d-6359-4b73-aeb4-75e93f468b9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:25.495001 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e86082d-6359-4b73-aeb4-75e93f468b9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:25.495275 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e86082d-6359-4b73-aeb4-75e93f468b9e admin admin] PATH_INFO: `/v3/users/914ae901011045fb81d83e02cceaea60` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:25.496536 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e86082d-6359-4b73-aeb4-75e93f468b9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=914ae901011045fb81d83e02cceaea60)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:25.498975 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e86082d-6359-4b73-aeb4-75e93f468b9e admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:25.536782 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d665d232-8e6c-4332-b769-ff18117613f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:25.537604 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d665d232-8e6c-4332-b769-ff18117613f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:25.537917 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d665d232-8e6c-4332-b769-ff18117613f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:25.538196 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d665d232-8e6c-4332-b769-ff18117613f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:25.539383 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d665d232-8e6c-4332-b769-ff18117613f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:25.541240 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d665d232-8e6c-4332-b769-ff18117613f5 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:25.552865 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 978/1966] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:30:25 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:30:25.553676 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d665d232-8e6c-4332-b769-ff18117613f5 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:25.565205 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d665d232-8e6c-4332-b769-ff18117613f5 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:25.570087 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8c042b-5504-4a15-8de8-a9f2df375b11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:25.570752 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8c042b-5504-4a15-8de8-a9f2df375b11 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:25.571035 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8c042b-5504-4a15-8de8-a9f2df375b11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:25.571323 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8c042b-5504-4a15-8de8-a9f2df375b11 admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:25.572021 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8c042b-5504-4a15-8de8-a9f2df375b11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:25.577673 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8c042b-5504-4a15-8de8-a9f2df375b11 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:25.686488 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 979/1967] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 17 19:30:25 2022] POST /identity/v3/users => generated 292 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:30:25.687327 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8c042b-5504-4a15-8de8-a9f2df375b11 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:25.696743 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8c042b-5504-4a15-8de8-a9f2df375b11 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:25.702239 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a27282-9d64-46de-a4df-92b06afead60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:25.702899 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a27282-9d64-46de-a4df-92b06afead60 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:25.703531 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a27282-9d64-46de-a4df-92b06afead60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:25.703531 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a27282-9d64-46de-a4df-92b06afead60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:25.704178 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a27282-9d64-46de-a4df-92b06afead60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:25.705872 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a27282-9d64-46de-a4df-92b06afead60 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:25.717023 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 980/1968] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:30:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:30:25.717781 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a27282-9d64-46de-a4df-92b06afead60 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:25.722828 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1e86082d-6359-4b73-aeb4-75e93f468b9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '914ae901011045fb81d83e02cceaea60'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:25.723714 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1e86082d-6359-4b73-aeb4-75e93f468b9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '914ae901011045fb81d83e02cceaea60'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:25.727633 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a27282-9d64-46de-a4df-92b06afead60 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:25.732669 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd66911-eace-4f9e-bff5-7d27a515e0ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:25.733543 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd66911-eace-4f9e-bff5-7d27a515e0ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:25.733823 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd66911-eace-4f9e-bff5-7d27a515e0ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:25.734093 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd66911-eace-4f9e-bff5-7d27a515e0ff admin admin] PATH_INFO: `/v3/projects/8cadb52387c240d09424f2b51a15a2aa/users/87e160fa85924c1e86880e44a41d35c5/roles/60af5088793e4379b88821a86552a344` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:25.737895 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1e86082d-6359-4b73-aeb4-75e93f468b9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '914ae901011045fb81d83e02cceaea60'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:25.754365 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd66911-eace-4f9e-bff5-7d27a515e0ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=8cadb52387c240d09424f2b51a15a2aa, user_id=87e160fa85924c1e86880e44a41d35c5, role_id=60af5088793e4379b88821a86552a344)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:25.760561 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd66911-eace-4f9e-bff5-7d27a515e0ff admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:25.816806 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1e86082d-6359-4b73-aeb4-75e93f468b9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '914ae901011045fb81d83e02cceaea60'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:25.824461 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1e86082d-6359-4b73-aeb4-75e93f468b9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '914ae901011045fb81d83e02cceaea60'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:25.831347 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 989/1969] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 17 19:30:25 2022] DELETE /identity/v3/users/914ae901011045fb81d83e02cceaea60 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:30:25.832057 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e86082d-6359-4b73-aeb4-75e93f468b9e admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:25.833353 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-1e86082d-6359-4b73-aeb4-75e93f468b9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:25.891125 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 981/1970] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:30:25 2022] PUT /identity/v3/projects/8cadb52387c240d09424f2b51a15a2aa/users/87e160fa85924c1e86880e44a41d35c5/roles/60af5088793e4379b88821a86552a344 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:30:25.892543 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd66911-eace-4f9e-bff5-7d27a515e0ff admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:25.896106 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-ecd66911-eace-4f9e-bff5-7d27a515e0ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:25.913584 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e86082d-6359-4b73-aeb4-75e93f468b9e admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:25.973442 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ad1415-a0d3-4451-a68b-39def1151705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:25.974209 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ad1415-a0d3-4451-a68b-39def1151705 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:25.974487 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ad1415-a0d3-4451-a68b-39def1151705 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:25.974807 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ad1415-a0d3-4451-a68b-39def1151705 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:25.975575 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ad1415-a0d3-4451-a68b-39def1151705 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:25.976940 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd66911-eace-4f9e-bff5-7d27a515e0ff admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:25.979816 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ad1415-a0d3-4451-a68b-39def1151705 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:25.982695 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ac1c75-3f52-482c-9503-6355dd0fb1a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:25.983783 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ac1c75-3f52-482c-9503-6355dd0fb1a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:25.984066 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ac1c75-3f52-482c-9503-6355dd0fb1a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:25.984341 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ac1c75-3f52-482c-9503-6355dd0fb1a6 admin admin] PATH_INFO: `/v3/projects/2ac6f7ce879c46c19105a26328674c31/users/aeb42c1e4b9b4ae58be990813a04ab74/roles/60af5088793e4379b88821a86552a344` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:25.989061 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 990/1971] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:30:25 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:30:25.990595 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d760c2-f4ff-47d5-9cc2-5b52fbd9a1ca None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:25.990775 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d760c2-f4ff-47d5-9cc2-5b52fbd9a1ca None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:25.991081 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d760c2-f4ff-47d5-9cc2-5b52fbd9a1ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:26.008671 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ac1c75-3f52-482c-9503-6355dd0fb1a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ac6f7ce879c46c19105a26328674c31, user_id=aeb42c1e4b9b4ae58be990813a04ab74, role_id=60af5088793e4379b88821a86552a344)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:26.009362 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ac1c75-3f52-482c-9503-6355dd0fb1a6 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:26.009811 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-58d760c2-f4ff-47d5-9cc2-5b52fbd9a1ca None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:30:26.011148 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-58d760c2-f4ff-47d5-9cc2-5b52fbd9a1ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:26.019064 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 991/1972] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:30:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:30:26.019981 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d760c2-f4ff-47d5-9cc2-5b52fbd9a1ca None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:26.021254 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-58d760c2-f4ff-47d5-9cc2-5b52fbd9a1ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:26.033753 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d760c2-f4ff-47d5-9cc2-5b52fbd9a1ca None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:26.038512 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ba33f4-9b55-4d98-9f94-6ad3a370b97f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:26.039182 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ba33f4-9b55-4d98-9f94-6ad3a370b97f admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:26.039460 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ba33f4-9b55-4d98-9f94-6ad3a370b97f admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:26.039764 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ba33f4-9b55-4d98-9f94-6ad3a370b97f admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:26.040495 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ba33f4-9b55-4d98-9f94-6ad3a370b97f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:26.042103 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ba33f4-9b55-4d98-9f94-6ad3a370b97f admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:26.053482 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 992/1973] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:30:26 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:30:26.054421 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ba33f4-9b55-4d98-9f94-6ad3a370b97f admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:26.065392 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ba33f4-9b55-4d98-9f94-6ad3a370b97f admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:26.070261 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c4b80f-fac3-4191-b833-5a671df97133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:26.070960 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c4b80f-fac3-4191-b833-5a671df97133 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:26.071472 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c4b80f-fac3-4191-b833-5a671df97133 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:26.071610 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c4b80f-fac3-4191-b833-5a671df97133 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:26.072259 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c4b80f-fac3-4191-b833-5a671df97133 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:26.073996 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c4b80f-fac3-4191-b833-5a671df97133 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:26.085255 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 993/1974] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:30:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:30:26.086425 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c4b80f-fac3-4191-b833-5a671df97133 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:26.088176 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-93c4b80f-fac3-4191-b833-5a671df97133 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:26.143046 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 982/1975] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:30:25 2022] PUT /identity/v3/projects/2ac6f7ce879c46c19105a26328674c31/users/aeb42c1e4b9b4ae58be990813a04ab74/roles/60af5088793e4379b88821a86552a344 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:30:26.144347 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ac1c75-3f52-482c-9503-6355dd0fb1a6 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:26.150590 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c4b80f-fac3-4191-b833-5a671df97133 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:26.204666 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a8b810-c8b5-4fd2-b103-bd54da1847ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:26.205960 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a8b810-c8b5-4fd2-b103-bd54da1847ec service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:26.206247 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a8b810-c8b5-4fd2-b103-bd54da1847ec service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:26.206521 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a8b810-c8b5-4fd2-b103-bd54da1847ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:26.218780 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ac1c75-3f52-482c-9503-6355dd0fb1a6 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:26.225298 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2ff4a8-f77f-4ca0-a35a-89cf3a1d6f63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:26.226190 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2ff4a8-f77f-4ca0-a35a-89cf3a1d6f63 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:26.226467 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2ff4a8-f77f-4ca0-a35a-89cf3a1d6f63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:26.226738 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2ff4a8-f77f-4ca0-a35a-89cf3a1d6f63 admin admin] PATH_INFO: `/v3/projects/8cadb52387c240d09424f2b51a15a2aa/users/87e160fa85924c1e86880e44a41d35c5/roles/60af5088793e4379b88821a86552a344` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:26.228765 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2ff4a8-f77f-4ca0-a35a-89cf3a1d6f63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8cadb52387c240d09424f2b51a15a2aa, user_id=87e160fa85924c1e86880e44a41d35c5, role_id=60af5088793e4379b88821a86552a344)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:26.232076 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2ff4a8-f77f-4ca0-a35a-89cf3a1d6f63 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:26.244579 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 983/1976] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:30:26 2022] PUT /identity/v3/projects/8cadb52387c240d09424f2b51a15a2aa/users/87e160fa85924c1e86880e44a41d35c5/roles/60af5088793e4379b88821a86552a344 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:30:26.245624 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2ff4a8-f77f-4ca0-a35a-89cf3a1d6f63 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:26.273425 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a8b810-c8b5-4fd2-b103-bd54da1847ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:26.274980 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a8b810-c8b5-4fd2-b103-bd54da1847ec service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:26.285456 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 994/1977] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:30:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:30:26.286204 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a8b810-c8b5-4fd2-b103-bd54da1847ec service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:26.297020 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a8b810-c8b5-4fd2-b103-bd54da1847ec service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:26.329856 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2ff4a8-f77f-4ca0-a35a-89cf3a1d6f63 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:26.334974 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89aef0c-be81-480b-9df2-a51ffbd17913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:26.336599 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89aef0c-be81-480b-9df2-a51ffbd17913 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:26.336940 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89aef0c-be81-480b-9df2-a51ffbd17913 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:26.339388 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89aef0c-be81-480b-9df2-a51ffbd17913 admin admin] PATH_INFO: `/v3/projects/19b750e65e6b43ffacd84c52ddba982d/users/8b4a841e3d3f4fa5bc000f828f1e9184/roles/114e6a16e63447d1924707ace5aace25` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:26.358661 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89aef0c-be81-480b-9df2-a51ffbd17913 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19b750e65e6b43ffacd84c52ddba982d, user_id=8b4a841e3d3f4fa5bc000f828f1e9184, role_id=114e6a16e63447d1924707ace5aace25)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:26.362435 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89aef0c-be81-480b-9df2-a51ffbd17913 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:26.381442 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f943d13-0f94-432e-a750-acd4facf2c07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:26.382560 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f943d13-0f94-432e-a750-acd4facf2c07 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:26.382946 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f943d13-0f94-432e-a750-acd4facf2c07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:26.383118 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f943d13-0f94-432e-a750-acd4facf2c07 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:26.383950 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f943d13-0f94-432e-a750-acd4facf2c07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:26.386562 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f943d13-0f94-432e-a750-acd4facf2c07 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:26.398222 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 995/1978] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:30:26 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:30:26.399708 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb55dec-0b38-40f5-b8d9-0416a7b5a4c5 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:26.399997 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb55dec-0b38-40f5-b8d9-0416a7b5a4c5 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:26.400589 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb55dec-0b38-40f5-b8d9-0416a7b5a4c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:26.441265 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-6fb55dec-0b38-40f5-b8d9-0416a7b5a4c5 None None] MFA Rules not processed for user `87e160fa85924c1e86880e44a41d35c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:30:26.442855 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-6fb55dec-0b38-40f5-b8d9-0416a7b5a4c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:26.501317 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 984/1979] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:30:26 2022] PUT /identity/v3/projects/19b750e65e6b43ffacd84c52ddba982d/users/8b4a841e3d3f4fa5bc000f828f1e9184/roles/114e6a16e63447d1924707ace5aace25 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:30:26.501926 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89aef0c-be81-480b-9df2-a51ffbd17913 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:26.563350 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 996/1980] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:30:26 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:30:26.569468 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd90ca3a-18ad-4cde-be71-a1009e41d188 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:26.569773 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd90ca3a-18ad-4cde-be71-a1009e41d188 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:26.570019 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd90ca3a-18ad-4cde-be71-a1009e41d188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:26.589555 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89aef0c-be81-480b-9df2-a51ffbd17913 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:26.603083 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-fd90ca3a-18ad-4cde-be71-a1009e41d188 None None] MFA Rules not processed for user `8b4a841e3d3f4fa5bc000f828f1e9184`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:30:26.604460 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-fd90ca3a-18ad-4cde-be71-a1009e41d188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:26.611131 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08177ffd-abdb-424e-a873-8735a629a24b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:26.612256 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08177ffd-abdb-424e-a873-8735a629a24b admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:26.612392 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08177ffd-abdb-424e-a873-8735a629a24b admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:26.612648 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08177ffd-abdb-424e-a873-8735a629a24b admin admin] PATH_INFO: `/v3/projects/2ac6f7ce879c46c19105a26328674c31/users/aeb42c1e4b9b4ae58be990813a04ab74/roles/60af5088793e4379b88821a86552a344` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:26.617080 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08177ffd-abdb-424e-a873-8735a629a24b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ac6f7ce879c46c19105a26328674c31, user_id=aeb42c1e4b9b4ae58be990813a04ab74, role_id=60af5088793e4379b88821a86552a344)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:26.620909 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08177ffd-abdb-424e-a873-8735a629a24b admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:26.666922 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 985/1981] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:30:26 2022] PUT /identity/v3/projects/2ac6f7ce879c46c19105a26328674c31/users/aeb42c1e4b9b4ae58be990813a04ab74/roles/60af5088793e4379b88821a86552a344 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:30:26.668084 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08177ffd-abdb-424e-a873-8735a629a24b admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:26.670741 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-08177ffd-abdb-424e-a873-8735a629a24b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:26.718400 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 997/1982] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:30:26 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:30:26.721919 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2434435c-3078-496a-b187-07201841266e None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:26.722303 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2434435c-3078-496a-b187-07201841266e None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:26.722713 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2434435c-3078-496a-b187-07201841266e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:26.741952 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08177ffd-abdb-424e-a873-8735a629a24b admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:26.746162 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa2e5f9-d592-480c-8505-02b7f461f6d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd05409b38ae346458abfe41ebda1c59e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:26.747948 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa2e5f9-d592-480c-8505-02b7f461f6d8 service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:26.748415 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa2e5f9-d592-480c-8505-02b7f461f6d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:26.748875 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa2e5f9-d592-480c-8505-02b7f461f6d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:26.762309 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-2434435c-3078-496a-b187-07201841266e None None] MFA Rules not processed for user `aeb42c1e4b9b4ae58be990813a04ab74`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:30:26.765237 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-2434435c-3078-496a-b187-07201841266e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:26.865419 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa2e5f9-d592-480c-8505-02b7f461f6d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:26.870220 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa2e5f9-d592-480c-8505-02b7f461f6d8 service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:26.903870 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 986/1983] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:30:26 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:30:26.905813 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa2e5f9-d592-480c-8505-02b7f461f6d8 service cinder] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:26.952614 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 998/1984] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:30:26 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:30:26.963670 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2434435c-3078-496a-b187-07201841266e None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:26.965097 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-2434435c-3078-496a-b187-07201841266e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:26.979385 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2434435c-3078-496a-b187-07201841266e None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:26.983465 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382ec022-340d-4a86-9b2d-fe49056c27d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd05409b38ae346458abfe41ebda1c59e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:26.984182 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382ec022-340d-4a86-9b2d-fe49056c27d6 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:26.984457 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382ec022-340d-4a86-9b2d-fe49056c27d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:26.985054 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382ec022-340d-4a86-9b2d-fe49056c27d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:27.002252 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382ec022-340d-4a86-9b2d-fe49056c27d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:27.002518 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382ec022-340d-4a86-9b2d-fe49056c27d6 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:27.013109 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 999/1985] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:30:26 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 17 19:30:27.015789 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa2e5f9-d592-480c-8505-02b7f461f6d8 service cinder] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:27.025263 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f0e432-bee3-4325-b632-239db4ecd801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:27.025263 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f0e432-bee3-4325-b632-239db4ecd801 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:27.025263 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f0e432-bee3-4325-b632-239db4ecd801 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:27.025263 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f0e432-bee3-4325-b632-239db4ecd801 admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:27.026078 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f0e432-bee3-4325-b632-239db4ecd801 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:27.030380 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f0e432-bee3-4325-b632-239db4ecd801 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:27.030522 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382ec022-340d-4a86-9b2d-fe49056c27d6 service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:27.034224 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-382ec022-340d-4a86-9b2d-fe49056c27d6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:27.081634 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382ec022-340d-4a86-9b2d-fe49056c27d6 service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:27.101580 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a58fe83-e3ca-4f32-9197-5d17c43d176e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:27.102713 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a58fe83-e3ca-4f32-9197-5d17c43d176e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:27.103134 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a58fe83-e3ca-4f32-9197-5d17c43d176e admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:27.103458 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a58fe83-e3ca-4f32-9197-5d17c43d176e admin admin] PATH_INFO: `/v3/projects/8dcf3d06de29437ebbd50e15eec83166` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:27.104984 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a58fe83-e3ca-4f32-9197-5d17c43d176e admin admin] RBAC: Authorizing `identity:delete_project(project_id=8dcf3d06de29437ebbd50e15eec83166)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:27.108361 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a58fe83-e3ca-4f32-9197-5d17c43d176e admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:27.148493 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 987/1986] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 17 19:30:26 2022] POST /identity/v3/projects => generated 383 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:30:27.149723 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f0e432-bee3-4325-b632-239db4ecd801 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:27.248280 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-5a58fe83-e3ca-4f32-9197-5d17c43d176e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8dcf3d06de29437ebbd50e15eec83166', 'request_id': 'req-5a58fe83-e3ca-4f32-9197-5d17c43d176e'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:27.257641 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-5a58fe83-e3ca-4f32-9197-5d17c43d176e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8dcf3d06de29437ebbd50e15eec83166', 'request_id': 'req-5a58fe83-e3ca-4f32-9197-5d17c43d176e'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:27.259689 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f0e432-bee3-4325-b632-239db4ecd801 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:27.321410 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0b9dfe-c27b-48e1-a763-c23cec6d7b04 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd05409b38ae346458abfe41ebda1c59e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:27.322067 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0b9dfe-c27b-48e1-a763-c23cec6d7b04 service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:27.324664 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0b9dfe-c27b-48e1-a763-c23cec6d7b04 service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:27.324664 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0b9dfe-c27b-48e1-a763-c23cec6d7b04 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:27.326110 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-9a0b9dfe-c27b-48e1-a763-c23cec6d7b04 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:27.370735 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-5a58fe83-e3ca-4f32-9197-5d17c43d176e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8dcf3d06de29437ebbd50e15eec83166', 'request_id': 'req-5a58fe83-e3ca-4f32-9197-5d17c43d176e'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:27.394987 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-5a58fe83-e3ca-4f32-9197-5d17c43d176e admin admin] The token cache is being invalidate because project 8dcf3d06de29437ebbd50e15eec83166 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 17 19:30:27.395403 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-5a58fe83-e3ca-4f32-9197-5d17c43d176e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:27.396315 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-5a58fe83-e3ca-4f32-9197-5d17c43d176e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:27.403635 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1000/1987] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 17 19:30:27 2022] DELETE /identity/v3/projects/8dcf3d06de29437ebbd50e15eec83166 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:30:27.403635 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a58fe83-e3ca-4f32-9197-5d17c43d176e admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:27.403635 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-5a58fe83-e3ca-4f32-9197-5d17c43d176e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:27.420895 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0b9dfe-c27b-48e1-a763-c23cec6d7b04 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:27.422527 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0b9dfe-c27b-48e1-a763-c23cec6d7b04 service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:27.443253 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 988/1988] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:30:27 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:30:27.446270 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0b9dfe-c27b-48e1-a763-c23cec6d7b04 service cinder] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:27.513389 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a58fe83-e3ca-4f32-9197-5d17c43d176e admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:27.520042 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebd4a8e-07a9-4e7a-9af8-4346ae47e56e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:27.520805 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebd4a8e-07a9-4e7a-9af8-4346ae47e56e admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:27.521164 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebd4a8e-07a9-4e7a-9af8-4346ae47e56e admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:27.521514 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebd4a8e-07a9-4e7a-9af8-4346ae47e56e admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:27.522280 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebd4a8e-07a9-4e7a-9af8-4346ae47e56e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:27.524404 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebd4a8e-07a9-4e7a-9af8-4346ae47e56e admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:27.531698 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0b9dfe-c27b-48e1-a763-c23cec6d7b04 service cinder] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:27.538959 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59e553e-44fd-4ca8-a446-3b5e6af59a4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:27.539956 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59e553e-44fd-4ca8-a446-3b5e6af59a4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:27.540235 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59e553e-44fd-4ca8-a446-3b5e6af59a4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:27.541936 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59e553e-44fd-4ca8-a446-3b5e6af59a4c admin admin] PATH_INFO: `/v3/users/13c35e4239514a60b42c16b1e216db87` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:27.543523 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59e553e-44fd-4ca8-a446-3b5e6af59a4c admin admin] RBAC: Authorizing `identity:delete_user(user_id=13c35e4239514a60b42c16b1e216db87)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:27.545506 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59e553e-44fd-4ca8-a446-3b5e6af59a4c admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:27.693924 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1001/1989] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 17 19:30:27 2022] POST /identity/v3/users => generated 298 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:30:27.702374 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebd4a8e-07a9-4e7a-9af8-4346ae47e56e admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:27.714328 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebd4a8e-07a9-4e7a-9af8-4346ae47e56e admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:27.719893 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f3aeda-0cd7-466c-8e7e-47db03d85ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:27.720695 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f3aeda-0cd7-466c-8e7e-47db03d85ea7 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:27.720981 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f3aeda-0cd7-466c-8e7e-47db03d85ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:27.721505 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f3aeda-0cd7-466c-8e7e-47db03d85ea7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:27.725292 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f3aeda-0cd7-466c-8e7e-47db03d85ea7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:27.725292 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f3aeda-0cd7-466c-8e7e-47db03d85ea7 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:27.734600 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1002/1990] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:30:27 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 17 19:30:27.740455 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f3aeda-0cd7-466c-8e7e-47db03d85ea7 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:27.753848 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f3aeda-0cd7-466c-8e7e-47db03d85ea7 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:27.760948 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb478aa9-6fd6-4e0b-afe1-9d8463b58378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:27.761842 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb478aa9-6fd6-4e0b-afe1-9d8463b58378 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:27.762121 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb478aa9-6fd6-4e0b-afe1-9d8463b58378 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:27.762946 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb478aa9-6fd6-4e0b-afe1-9d8463b58378 admin admin] PATH_INFO: `/v3/projects/4ac95786705c46e0b49273629304d0d4/users/fc2cb586d2eb45bb966a010bd3ee50e2/roles/60af5088793e4379b88821a86552a344` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:27.778382 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb478aa9-6fd6-4e0b-afe1-9d8463b58378 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ac95786705c46e0b49273629304d0d4, user_id=fc2cb586d2eb45bb966a010bd3ee50e2, role_id=60af5088793e4379b88821a86552a344)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:27.781131 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb478aa9-6fd6-4e0b-afe1-9d8463b58378 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:27.798528 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-f59e553e-44fd-4ca8-a446-3b5e6af59a4c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13c35e4239514a60b42c16b1e216db87'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:27.804299 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-f59e553e-44fd-4ca8-a446-3b5e6af59a4c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13c35e4239514a60b42c16b1e216db87'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:27.818320 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-f59e553e-44fd-4ca8-a446-3b5e6af59a4c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13c35e4239514a60b42c16b1e216db87'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:27.819050 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-f59e553e-44fd-4ca8-a446-3b5e6af59a4c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13c35e4239514a60b42c16b1e216db87'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:27.819700 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-f59e553e-44fd-4ca8-a446-3b5e6af59a4c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13c35e4239514a60b42c16b1e216db87'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:27.858768 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1003/1991] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:30:27 2022] PUT /identity/v3/projects/4ac95786705c46e0b49273629304d0d4/users/fc2cb586d2eb45bb966a010bd3ee50e2/roles/60af5088793e4379b88821a86552a344 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:30:27.862386 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb478aa9-6fd6-4e0b-afe1-9d8463b58378 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:27.864212 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-eb478aa9-6fd6-4e0b-afe1-9d8463b58378 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:27.889093 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 989/1992] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 17 19:30:27 2022] DELETE /identity/v3/users/13c35e4239514a60b42c16b1e216db87 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:30:27.974596 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb478aa9-6fd6-4e0b-afe1-9d8463b58378 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:28.053850 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f099dd11-7131-4ea1-98dc-922e09f9364e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:28.054446 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f099dd11-7131-4ea1-98dc-922e09f9364e admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:28.054728 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f099dd11-7131-4ea1-98dc-922e09f9364e admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:28.055001 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f099dd11-7131-4ea1-98dc-922e09f9364e admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:28.055912 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f099dd11-7131-4ea1-98dc-922e09f9364e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:28.057779 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f099dd11-7131-4ea1-98dc-922e09f9364e admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:28.072141 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1004/1993] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:30:27 2022] GET /identity/v3/roles => generated 1420 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 17 19:30:28.076966 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59e553e-44fd-4ca8-a446-3b5e6af59a4c admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:28.088557 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59e553e-44fd-4ca8-a446-3b5e6af59a4c admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:28.097723 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11edde7f-abaf-4336-99dd-7a4f2570079d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:28.098849 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11edde7f-abaf-4336-99dd-7a4f2570079d admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:28.099224 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11edde7f-abaf-4336-99dd-7a4f2570079d admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:28.099640 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11edde7f-abaf-4336-99dd-7a4f2570079d admin admin] PATH_INFO: `/v3/projects/4ac95786705c46e0b49273629304d0d4/users/fc2cb586d2eb45bb966a010bd3ee50e2/roles/60af5088793e4379b88821a86552a344` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:28.101734 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11edde7f-abaf-4336-99dd-7a4f2570079d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ac95786705c46e0b49273629304d0d4, user_id=fc2cb586d2eb45bb966a010bd3ee50e2, role_id=60af5088793e4379b88821a86552a344)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:28.104515 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11edde7f-abaf-4336-99dd-7a4f2570079d admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:28.120192 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 990/1994] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:30:28 2022] PUT /identity/v3/projects/4ac95786705c46e0b49273629304d0d4/users/fc2cb586d2eb45bb966a010bd3ee50e2/roles/60af5088793e4379b88821a86552a344 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 17 19:30:28.131636 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f483853-5740-4fb1-b154-a8a5ae51a270 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:28.131857 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f483853-5740-4fb1-b154-a8a5ae51a270 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:28.132134 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f483853-5740-4fb1-b154-a8a5ae51a270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:28.163633 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-2f483853-5740-4fb1-b154-a8a5ae51a270 None None] MFA Rules not processed for user `fc2cb586d2eb45bb966a010bd3ee50e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:30:28.164879 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-2f483853-5740-4fb1-b154-a8a5ae51a270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:28.241951 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1005/1995] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:30:28 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:30:28.388877 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11edde7f-abaf-4336-99dd-7a4f2570079d admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:28.390559 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-11edde7f-abaf-4336-99dd-7a4f2570079d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:28.429693 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f483853-5740-4fb1-b154-a8a5ae51a270 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:28.431580 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-2f483853-5740-4fb1-b154-a8a5ae51a270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:28.494133 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11edde7f-abaf-4336-99dd-7a4f2570079d admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:28.499585 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a94c2a2-93f3-4d26-9c5b-da83ca5ef482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:28.503386 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a94c2a2-93f3-4d26-9c5b-da83ca5ef482 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:28.503386 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a94c2a2-93f3-4d26-9c5b-da83ca5ef482 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:28.503386 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a94c2a2-93f3-4d26-9c5b-da83ca5ef482 admin admin] PATH_INFO: `/v3/users/6a537695d7c241808534934d76c58deb` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:28.503386 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a94c2a2-93f3-4d26-9c5b-da83ca5ef482 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a537695d7c241808534934d76c58deb)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:28.504196 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a94c2a2-93f3-4d26-9c5b-da83ca5ef482 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:28.529768 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f483853-5740-4fb1-b154-a8a5ae51a270 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:28.535051 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1aa3f92-ed4d-420c-ae1d-1c7e6e822afc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:28.537868 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aa3f92-ed4d-420c-ae1d-1c7e6e822afc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:28.537868 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aa3f92-ed4d-420c-ae1d-1c7e6e822afc admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:28.537868 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aa3f92-ed4d-420c-ae1d-1c7e6e822afc admin admin] PATH_INFO: `/v3/projects/cc489143a7284b8fb1b691c2f666a361` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:28.537868 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1aa3f92-ed4d-420c-ae1d-1c7e6e822afc admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc489143a7284b8fb1b691c2f666a361)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:28.539636 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1aa3f92-ed4d-420c-ae1d-1c7e6e822afc admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:28.731844 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-5a94c2a2-93f3-4d26-9c5b-da83ca5ef482 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a537695d7c241808534934d76c58deb'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:28.737730 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c1aa3f92-ed4d-420c-ae1d-1c7e6e822afc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc489143a7284b8fb1b691c2f666a361', 'request_id': 'req-c1aa3f92-ed4d-420c-ae1d-1c7e6e822afc'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:28.739883 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-5a94c2a2-93f3-4d26-9c5b-da83ca5ef482 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a537695d7c241808534934d76c58deb'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:28.748000 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c1aa3f92-ed4d-420c-ae1d-1c7e6e822afc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc489143a7284b8fb1b691c2f666a361', 'request_id': 'req-c1aa3f92-ed4d-420c-ae1d-1c7e6e822afc'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:28.753201 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-5a94c2a2-93f3-4d26-9c5b-da83ca5ef482 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a537695d7c241808534934d76c58deb'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:28.758492 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-5a94c2a2-93f3-4d26-9c5b-da83ca5ef482 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a537695d7c241808534934d76c58deb'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:28.758492 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-5a94c2a2-93f3-4d26-9c5b-da83ca5ef482 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a537695d7c241808534934d76c58deb'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:28.833809 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c1aa3f92-ed4d-420c-ae1d-1c7e6e822afc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc489143a7284b8fb1b691c2f666a361', 'request_id': 'req-c1aa3f92-ed4d-420c-ae1d-1c7e6e822afc'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:28.845251 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c1aa3f92-ed4d-420c-ae1d-1c7e6e822afc admin admin] The token cache is being invalidate because project cc489143a7284b8fb1b691c2f666a361 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 17 19:30:28.845533 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c1aa3f92-ed4d-420c-ae1d-1c7e6e822afc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:28.846408 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c1aa3f92-ed4d-420c-ae1d-1c7e6e822afc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:28.848050 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1006/1996] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 17 19:30:28 2022] DELETE /identity/v3/projects/cc489143a7284b8fb1b691c2f666a361 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 17 19:30:28.848805 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1aa3f92-ed4d-420c-ae1d-1c7e6e822afc admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:28.850089 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c1aa3f92-ed4d-420c-ae1d-1c7e6e822afc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:28.916875 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1aa3f92-ed4d-420c-ae1d-1c7e6e822afc admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:28.921514 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541711f5-0bc5-42a8-86bb-b9f936c843f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd05409b38ae346458abfe41ebda1c59e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:28.922169 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541711f5-0bc5-42a8-86bb-b9f936c843f4 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:28.922445 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541711f5-0bc5-42a8-86bb-b9f936c843f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:28.922730 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541711f5-0bc5-42a8-86bb-b9f936c843f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:28.924690 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-541711f5-0bc5-42a8-86bb-b9f936c843f4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:28.942304 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 991/1997] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 17 19:30:28 2022] DELETE /identity/v3/users/6a537695d7c241808534934d76c58deb => generated 0 bytes in 554 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 17 19:30:28.946963 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bd19df-fd9c-4eac-b146-924524de9492 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:28.951108 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bd19df-fd9c-4eac-b146-924524de9492 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:28.951108 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bd19df-fd9c-4eac-b146-924524de9492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:28.975966 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-b1bd19df-fd9c-4eac-b146-924524de9492 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:30:28.977175 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-b1bd19df-fd9c-4eac-b146-924524de9492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:29.000421 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541711f5-0bc5-42a8-86bb-b9f936c843f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:29.002131 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541711f5-0bc5-42a8-86bb-b9f936c843f4 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:29.019800 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1007/1998] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:30:28 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:30:29.021361 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1f148c-ae37-4729-97c2-7ed0aac3cfc8 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:29.021554 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1f148c-ae37-4729-97c2-7ed0aac3cfc8 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:29.021835 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1f148c-ae37-4729-97c2-7ed0aac3cfc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:29.041208 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-7c1f148c-ae37-4729-97c2-7ed0aac3cfc8 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:30:29.042423 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-7c1f148c-ae37-4729-97c2-7ed0aac3cfc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:29.045026 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 992/1999] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:30:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:30:29.055699 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1008/2000] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:30:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:30:29.055880 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a951e730-ecd6-42c7-98e3-b78ab4035876 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:29.055880 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a951e730-ecd6-42c7-98e3-b78ab4035876 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:29.055880 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a951e730-ecd6-42c7-98e3-b78ab4035876 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:29.065657 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1f148c-ae37-4729-97c2-7ed0aac3cfc8 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:29.069722 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-7c1f148c-ae37-4729-97c2-7ed0aac3cfc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:29.077400 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-a951e730-ecd6-42c7-98e3-b78ab4035876 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:30:29.078698 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-a951e730-ecd6-42c7-98e3-b78ab4035876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:29.089561 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 993/2001] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:30:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:30:29.093943 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a951e730-ecd6-42c7-98e3-b78ab4035876 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:29.103370 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a951e730-ecd6-42c7-98e3-b78ab4035876 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:29.111824 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18462a9-dd54-4f47-8a4f-f535eb5fe8f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:29.113798 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18462a9-dd54-4f47-8a4f-f535eb5fe8f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:29.113798 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18462a9-dd54-4f47-8a4f-f535eb5fe8f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:29.114004 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18462a9-dd54-4f47-8a4f-f535eb5fe8f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:29.114755 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18462a9-dd54-4f47-8a4f-f535eb5fe8f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:30:29.115067 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18462a9-dd54-4f47-8a4f-f535eb5fe8f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:29.116774 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18462a9-dd54-4f47-8a4f-f535eb5fe8f7 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:29.129924 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 994/2002] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 17 19:30:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:30:29.133836 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18462a9-dd54-4f47-8a4f-f535eb5fe8f7 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:29.145962 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18462a9-dd54-4f47-8a4f-f535eb5fe8f7 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:29.147578 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1f148c-ae37-4729-97c2-7ed0aac3cfc8 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:29.151822 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0379995a-0028-4c79-8394-25a7e41c5a7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:29.152325 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0379995a-0028-4c79-8394-25a7e41c5a7f service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:29.154155 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a5e9be-90d2-4fc6-a271-a15972e02693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:29.154309 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0379995a-0028-4c79-8394-25a7e41c5a7f service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:29.154597 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0379995a-0028-4c79-8394-25a7e41c5a7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:29.154842 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a5e9be-90d2-4fc6-a271-a15972e02693 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:29.155093 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a5e9be-90d2-4fc6-a271-a15972e02693 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:29.155371 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a5e9be-90d2-4fc6-a271-a15972e02693 admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:29.156094 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a5e9be-90d2-4fc6-a271-a15972e02693 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:29.157991 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a5e9be-90d2-4fc6-a271-a15972e02693 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:29.174485 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0379995a-0028-4c79-8394-25a7e41c5a7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:29.175999 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0379995a-0028-4c79-8394-25a7e41c5a7f service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:29.189574 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1009/2003] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:30:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 17 19:30:29.200621 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0379995a-0028-4c79-8394-25a7e41c5a7f service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:29.251797 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 995/2004] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 17 19:30:29 2022] POST /identity/v3/projects => generated 373 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 17 19:30:29.256822 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a5e9be-90d2-4fc6-a271-a15972e02693 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:29.311857 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0379995a-0028-4c79-8394-25a7e41c5a7f service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:29.373311 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a5e9be-90d2-4fc6-a271-a15972e02693 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:29.379751 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82ceb98-3c63-4826-a518-040482ede5a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:29.382214 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82ceb98-3c63-4826-a518-040482ede5a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:29.382214 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82ceb98-3c63-4826-a518-040482ede5a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:29.382214 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82ceb98-3c63-4826-a518-040482ede5a1 admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:29.382214 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82ceb98-3c63-4826-a518-040482ede5a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:29.383620 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82ceb98-3c63-4826-a518-040482ede5a1 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:29.388494 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12d9b1d-b8d0-46e1-99d3-b006772000f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd05409b38ae346458abfe41ebda1c59e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:29.389271 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12d9b1d-b8d0-46e1-99d3-b006772000f1 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:29.389449 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12d9b1d-b8d0-46e1-99d3-b006772000f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:29.389747 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12d9b1d-b8d0-46e1-99d3-b006772000f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:29.391594 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a12d9b1d-b8d0-46e1-99d3-b006772000f1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:29.457216 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12d9b1d-b8d0-46e1-99d3-b006772000f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:29.459182 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12d9b1d-b8d0-46e1-99d3-b006772000f1 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:29.476769 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1010/2005] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:30:29 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:30:29.531456 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 996/2006] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 17 19:30:29 2022] POST /identity/v3/users => generated 287 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:30:29.532865 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12d9b1d-b8d0-46e1-99d3-b006772000f1 service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:29.548157 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12d9b1d-b8d0-46e1-99d3-b006772000f1 service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:29.553883 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d18135d-46e5-4f12-9959-5752bec04c57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:29.554550 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d18135d-46e5-4f12-9959-5752bec04c57 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:29.554830 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d18135d-46e5-4f12-9959-5752bec04c57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:29.555684 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d18135d-46e5-4f12-9959-5752bec04c57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:29.556422 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d18135d-46e5-4f12-9959-5752bec04c57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:29.558699 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d18135d-46e5-4f12-9959-5752bec04c57 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:29.570231 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1011/2007] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:30:29 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:30:29.574275 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82ceb98-3c63-4826-a518-040482ede5a1 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:29.586055 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82ceb98-3c63-4826-a518-040482ede5a1 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:29.590959 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc0b8ab-fa1d-48fa-8109-56de5c228362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:29.591665 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc0b8ab-fa1d-48fa-8109-56de5c228362 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:29.591950 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc0b8ab-fa1d-48fa-8109-56de5c228362 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:29.592223 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc0b8ab-fa1d-48fa-8109-56de5c228362 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:29.593107 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc0b8ab-fa1d-48fa-8109-56de5c228362 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:29.594719 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc0b8ab-fa1d-48fa-8109-56de5c228362 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:29.609359 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 997/2008] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:30:29 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:30:29.613443 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d18135d-46e5-4f12-9959-5752bec04c57 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:29.631651 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d18135d-46e5-4f12-9959-5752bec04c57 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:29.641829 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10426bb5-d0cc-4782-b228-c93de0a12745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:29.642815 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10426bb5-d0cc-4782-b228-c93de0a12745 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:29.643169 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10426bb5-d0cc-4782-b228-c93de0a12745 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:29.643619 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10426bb5-d0cc-4782-b228-c93de0a12745 admin admin] PATH_INFO: `/v3/projects/f18adcec58b14844bec04657953b5d10/users/e0d5404a4851455d9b82a9f21a641625/roles/114e6a16e63447d1924707ace5aace25` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:29.663471 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10426bb5-d0cc-4782-b228-c93de0a12745 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f18adcec58b14844bec04657953b5d10, user_id=e0d5404a4851455d9b82a9f21a641625, role_id=114e6a16e63447d1924707ace5aace25)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:29.666209 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10426bb5-d0cc-4782-b228-c93de0a12745 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:29.722520 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc0b8ab-fa1d-48fa-8109-56de5c228362 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:29.725317 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-2fc0b8ab-fa1d-48fa-8109-56de5c228362 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:29.753795 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc0b8ab-fa1d-48fa-8109-56de5c228362 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:29.759070 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1012/2009] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:30:29 2022] PUT /identity/v3/projects/f18adcec58b14844bec04657953b5d10/users/e0d5404a4851455d9b82a9f21a641625/roles/114e6a16e63447d1924707ace5aace25 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:30:29.766046 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f672d6-ef39-41c2-8cd7-84ae62907ba4 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:29.766345 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f672d6-ef39-41c2-8cd7-84ae62907ba4 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:29.766622 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f672d6-ef39-41c2-8cd7-84ae62907ba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:29.799912 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-03f672d6-ef39-41c2-8cd7-84ae62907ba4 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:30:29.801976 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-03f672d6-ef39-41c2-8cd7-84ae62907ba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:29.849770 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de13c2cf-bcfd-4a85-a9bb-5866aedaf995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:29.851228 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de13c2cf-bcfd-4a85-a9bb-5866aedaf995 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:29.851612 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de13c2cf-bcfd-4a85-a9bb-5866aedaf995 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:29.851953 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de13c2cf-bcfd-4a85-a9bb-5866aedaf995 admin admin] PATH_INFO: `/v3/projects/bfe4515ffcc14ccc81e01855fc4a5309` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:29.853341 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de13c2cf-bcfd-4a85-a9bb-5866aedaf995 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bfe4515ffcc14ccc81e01855fc4a5309)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:29.855396 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de13c2cf-bcfd-4a85-a9bb-5866aedaf995 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:29.882338 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1013/2010] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:30:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:30:29.883766 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f672d6-ef39-41c2-8cd7-84ae62907ba4 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:29.947580 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f672d6-ef39-41c2-8cd7-84ae62907ba4 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:29.951697 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045d098a-3dbe-4d37-ba15-965b403cdab0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd05409b38ae346458abfe41ebda1c59e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:29.952382 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045d098a-3dbe-4d37-ba15-965b403cdab0 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:29.952712 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045d098a-3dbe-4d37-ba15-965b403cdab0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:29.953032 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045d098a-3dbe-4d37-ba15-965b403cdab0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:29.954835 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-045d098a-3dbe-4d37-ba15-965b403cdab0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:30.023070 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045d098a-3dbe-4d37-ba15-965b403cdab0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:30.024904 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045d098a-3dbe-4d37-ba15-965b403cdab0 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:30.033853 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-de13c2cf-bcfd-4a85-a9bb-5866aedaf995 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bfe4515ffcc14ccc81e01855fc4a5309', 'request_id': 'req-de13c2cf-bcfd-4a85-a9bb-5866aedaf995'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:30.036068 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1014/2011] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:30:29 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:30:30.038812 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045d098a-3dbe-4d37-ba15-965b403cdab0 service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:30.041448 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-de13c2cf-bcfd-4a85-a9bb-5866aedaf995 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bfe4515ffcc14ccc81e01855fc4a5309', 'request_id': 'req-de13c2cf-bcfd-4a85-a9bb-5866aedaf995'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:30.048089 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-de13c2cf-bcfd-4a85-a9bb-5866aedaf995 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bfe4515ffcc14ccc81e01855fc4a5309', 'request_id': 'req-de13c2cf-bcfd-4a85-a9bb-5866aedaf995'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:30.103914 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045d098a-3dbe-4d37-ba15-965b403cdab0 service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:30.161154 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35be1166-5d5a-44ce-9f61-829d339d047b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:30.161815 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35be1166-5d5a-44ce-9f61-829d339d047b service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:30.162097 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35be1166-5d5a-44ce-9f61-829d339d047b service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:30.162371 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35be1166-5d5a-44ce-9f61-829d339d047b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:30.171807 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-de13c2cf-bcfd-4a85-a9bb-5866aedaf995 admin admin] The token cache is being invalidate because project bfe4515ffcc14ccc81e01855fc4a5309 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 17 19:30:30.172315 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-de13c2cf-bcfd-4a85-a9bb-5866aedaf995 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:30.173128 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-de13c2cf-bcfd-4a85-a9bb-5866aedaf995 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:30.178202 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 998/2012] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 17 19:30:29 2022] DELETE /identity/v3/projects/bfe4515ffcc14ccc81e01855fc4a5309 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:30:30.179273 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de13c2cf-bcfd-4a85-a9bb-5866aedaf995 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:30.180675 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-de13c2cf-bcfd-4a85-a9bb-5866aedaf995 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:30.245675 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35be1166-5d5a-44ce-9f61-829d339d047b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:30.248272 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35be1166-5d5a-44ce-9f61-829d339d047b service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:30.258071 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1015/2013] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:30:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:30:30.269010 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de13c2cf-bcfd-4a85-a9bb-5866aedaf995 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:30.274456 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bab7485-3898-4382-8eac-5bb59320de5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:30.276879 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bab7485-3898-4382-8eac-5bb59320de5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:30.276879 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bab7485-3898-4382-8eac-5bb59320de5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:30.276879 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bab7485-3898-4382-8eac-5bb59320de5c admin admin] PATH_INFO: `/v3/users/aeb42c1e4b9b4ae58be990813a04ab74` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:30.277113 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bab7485-3898-4382-8eac-5bb59320de5c admin admin] RBAC: Authorizing `identity:delete_user(user_id=aeb42c1e4b9b4ae58be990813a04ab74)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:30.279620 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bab7485-3898-4382-8eac-5bb59320de5c admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:30.328250 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35be1166-5d5a-44ce-9f61-829d339d047b service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:30.329783 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-35be1166-5d5a-44ce-9f61-829d339d047b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:30.343284 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35be1166-5d5a-44ce-9f61-829d339d047b service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:30.348857 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6638d606-ff83-4c8c-8435-a3ef4aec7c35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:30.349721 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6638d606-ff83-4c8c-8435-a3ef4aec7c35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:30.350021 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6638d606-ff83-4c8c-8435-a3ef4aec7c35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:30.350293 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6638d606-ff83-4c8c-8435-a3ef4aec7c35 admin admin] PATH_INFO: `/v3/users/8b4a841e3d3f4fa5bc000f828f1e9184` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:30.351776 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6638d606-ff83-4c8c-8435-a3ef4aec7c35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b4a841e3d3f4fa5bc000f828f1e9184)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:30.353754 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6638d606-ff83-4c8c-8435-a3ef4aec7c35 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:30.507584 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-7bab7485-3898-4382-8eac-5bb59320de5c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aeb42c1e4b9b4ae58be990813a04ab74'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:30.515722 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-7bab7485-3898-4382-8eac-5bb59320de5c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aeb42c1e4b9b4ae58be990813a04ab74'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:30.544533 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-7bab7485-3898-4382-8eac-5bb59320de5c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aeb42c1e4b9b4ae58be990813a04ab74'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:30.545385 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-7bab7485-3898-4382-8eac-5bb59320de5c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aeb42c1e4b9b4ae58be990813a04ab74'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:30.546242 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-7bab7485-3898-4382-8eac-5bb59320de5c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aeb42c1e4b9b4ae58be990813a04ab74'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:30.557801 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-6638d606-ff83-4c8c-8435-a3ef4aec7c35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b4a841e3d3f4fa5bc000f828f1e9184'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:30.558122 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-6638d606-ff83-4c8c-8435-a3ef4aec7c35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b4a841e3d3f4fa5bc000f828f1e9184'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:30.579055 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-6638d606-ff83-4c8c-8435-a3ef4aec7c35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b4a841e3d3f4fa5bc000f828f1e9184'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:30.676773 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 999/2014] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 17 19:30:30 2022] DELETE /identity/v3/users/aeb42c1e4b9b4ae58be990813a04ab74 => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:30:30.765536 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-6638d606-ff83-4c8c-8435-a3ef4aec7c35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b4a841e3d3f4fa5bc000f828f1e9184'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:30.773583 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-6638d606-ff83-4c8c-8435-a3ef4aec7c35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b4a841e3d3f4fa5bc000f828f1e9184'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:30.792991 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1016/2015] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 17 19:30:30 2022] DELETE /identity/v3/users/8b4a841e3d3f4fa5bc000f828f1e9184 => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 17 19:30:30.796846 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e92799-6599-468a-887b-27f3320d8bfe None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:30.797134 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e92799-6599-468a-887b-27f3320d8bfe None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:30.797432 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e92799-6599-468a-887b-27f3320d8bfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:30.827177 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-84e92799-6599-468a-887b-27f3320d8bfe None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:30:30.833325 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-84e92799-6599-468a-887b-27f3320d8bfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:30.907880 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1000/2016] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:30:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:30:30.914482 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6638d606-ff83-4c8c-8435-a3ef4aec7c35 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:30.916046 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-6638d606-ff83-4c8c-8435-a3ef4aec7c35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:31.018572 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6638d606-ff83-4c8c-8435-a3ef4aec7c35 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:31.026148 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab6cd94-a9b9-4442-ba3f-e8daf7e920cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:31.028692 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab6cd94-a9b9-4442-ba3f-e8daf7e920cb service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:31.028692 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab6cd94-a9b9-4442-ba3f-e8daf7e920cb service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:31.029249 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab6cd94-a9b9-4442-ba3f-e8daf7e920cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:31.041524 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab6cd94-a9b9-4442-ba3f-e8daf7e920cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:31.043260 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab6cd94-a9b9-4442-ba3f-e8daf7e920cb service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:31.054986 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1017/2017] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:30:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:30:31.324032 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e92799-6599-468a-887b-27f3320d8bfe None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:31.326257 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-84e92799-6599-468a-887b-27f3320d8bfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:31.345850 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e92799-6599-468a-887b-27f3320d8bfe None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:31.363442 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011a8d93-e947-40d8-90bb-9a4ac4a581d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:31.365345 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011a8d93-e947-40d8-90bb-9a4ac4a581d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:31.367614 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011a8d93-e947-40d8-90bb-9a4ac4a581d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:31.368050 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011a8d93-e947-40d8-90bb-9a4ac4a581d3 admin admin] PATH_INFO: `/v3/projects/2ac6f7ce879c46c19105a26328674c31` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:31.374461 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011a8d93-e947-40d8-90bb-9a4ac4a581d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ac6f7ce879c46c19105a26328674c31)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:31.376679 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011a8d93-e947-40d8-90bb-9a4ac4a581d3 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:31.533931 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-011a8d93-e947-40d8-90bb-9a4ac4a581d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ac6f7ce879c46c19105a26328674c31', 'request_id': 'req-011a8d93-e947-40d8-90bb-9a4ac4a581d3'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:31.540698 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-011a8d93-e947-40d8-90bb-9a4ac4a581d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ac6f7ce879c46c19105a26328674c31', 'request_id': 'req-011a8d93-e947-40d8-90bb-9a4ac4a581d3'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:31.546507 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-011a8d93-e947-40d8-90bb-9a4ac4a581d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ac6f7ce879c46c19105a26328674c31', 'request_id': 'req-011a8d93-e947-40d8-90bb-9a4ac4a581d3'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:31.736557 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab6cd94-a9b9-4442-ba3f-e8daf7e920cb service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:31.738011 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-7ab6cd94-a9b9-4442-ba3f-e8daf7e920cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:31.743650 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-011a8d93-e947-40d8-90bb-9a4ac4a581d3 admin admin] The token cache is being invalidate because project 2ac6f7ce879c46c19105a26328674c31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 17 19:30:31.743650 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-011a8d93-e947-40d8-90bb-9a4ac4a581d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:31.743650 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-011a8d93-e947-40d8-90bb-9a4ac4a581d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:31.743650 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1001/2018] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 17 19:30:31 2022] DELETE /identity/v3/projects/2ac6f7ce879c46c19105a26328674c31 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:30:31.749746 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b7ca45-0ef0-4a56-b731-445f5211dd96 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:31.749746 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b7ca45-0ef0-4a56-b731-445f5211dd96 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:31.749746 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b7ca45-0ef0-4a56-b731-445f5211dd96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:31.761743 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab6cd94-a9b9-4442-ba3f-e8daf7e920cb service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:31.833816 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-e8b7ca45-0ef0-4a56-b731-445f5211dd96 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:30:31.835106 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-e8b7ca45-0ef0-4a56-b731-445f5211dd96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:31.886432 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794a8cd7-97c8-4391-a7b6-c57571d835a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:31.887858 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794a8cd7-97c8-4391-a7b6-c57571d835a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:31.888141 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794a8cd7-97c8-4391-a7b6-c57571d835a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:31.888420 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794a8cd7-97c8-4391-a7b6-c57571d835a2 admin admin] PATH_INFO: `/v3/projects/19b750e65e6b43ffacd84c52ddba982d` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:31.889738 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794a8cd7-97c8-4391-a7b6-c57571d835a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19b750e65e6b43ffacd84c52ddba982d)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:31.892880 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794a8cd7-97c8-4391-a7b6-c57571d835a2 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:31.940381 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1002/2019] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:30:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:30:31.953694 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62550ece-63dd-48b9-be8a-74503f9610a2 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:31.954775 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62550ece-63dd-48b9-be8a-74503f9610a2 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:31.955079 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62550ece-63dd-48b9-be8a-74503f9610a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:31.976800 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-62550ece-63dd-48b9-be8a-74503f9610a2 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:30:31.977991 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-62550ece-63dd-48b9-be8a-74503f9610a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:31.988558 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1003/2020] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:30:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:30:31.992791 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62550ece-63dd-48b9-be8a-74503f9610a2 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:32.002868 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62550ece-63dd-48b9-be8a-74503f9610a2 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:32.010524 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df2a4af-1354-4a22-89f9-fc75a0744ac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:32.011198 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df2a4af-1354-4a22-89f9-fc75a0744ac9 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:32.011480 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df2a4af-1354-4a22-89f9-fc75a0744ac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:32.011790 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df2a4af-1354-4a22-89f9-fc75a0744ac9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:32.013056 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df2a4af-1354-4a22-89f9-fc75a0744ac9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:30:32.013362 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df2a4af-1354-4a22-89f9-fc75a0744ac9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:32.014989 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df2a4af-1354-4a22-89f9-fc75a0744ac9 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:32.030005 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1004/2021] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 17 19:30:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:30:32.038158 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df2a4af-1354-4a22-89f9-fc75a0744ac9 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:32.053720 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df2a4af-1354-4a22-89f9-fc75a0744ac9 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:32.082069 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-794a8cd7-97c8-4391-a7b6-c57571d835a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19b750e65e6b43ffacd84c52ddba982d', 'request_id': 'req-794a8cd7-97c8-4391-a7b6-c57571d835a2'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:32.089653 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-794a8cd7-97c8-4391-a7b6-c57571d835a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19b750e65e6b43ffacd84c52ddba982d', 'request_id': 'req-794a8cd7-97c8-4391-a7b6-c57571d835a2'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:32.127957 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a689667b-9a50-4177-bb88-8d46524804cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:32.128912 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a689667b-9a50-4177-bb88-8d46524804cd admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:32.129299 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a689667b-9a50-4177-bb88-8d46524804cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:32.129671 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a689667b-9a50-4177-bb88-8d46524804cd admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:32.130521 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a689667b-9a50-4177-bb88-8d46524804cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:32.132573 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a689667b-9a50-4177-bb88-8d46524804cd admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:32.272121 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-794a8cd7-97c8-4391-a7b6-c57571d835a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19b750e65e6b43ffacd84c52ddba982d', 'request_id': 'req-794a8cd7-97c8-4391-a7b6-c57571d835a2'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:32.291987 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-794a8cd7-97c8-4391-a7b6-c57571d835a2 admin admin] The token cache is being invalidate because project 19b750e65e6b43ffacd84c52ddba982d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 17 19:30:32.292280 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-794a8cd7-97c8-4391-a7b6-c57571d835a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:32.292962 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-794a8cd7-97c8-4391-a7b6-c57571d835a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:32.294724 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1018/2022] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 17 19:30:31 2022] DELETE /identity/v3/projects/19b750e65e6b43ffacd84c52ddba982d => generated 0 bytes in 559 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 17 19:30:32.296005 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6dd5e0-3e58-4f39-bbd4-9603aca82ba2 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:32.296286 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6dd5e0-3e58-4f39-bbd4-9603aca82ba2 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:32.297552 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6dd5e0-3e58-4f39-bbd4-9603aca82ba2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:32.309931 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1005/2023] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 17 19:30:32 2022] POST /identity/v3/projects => generated 377 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:30:32.311069 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a689667b-9a50-4177-bb88-8d46524804cd admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:32.313010 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-a689667b-9a50-4177-bb88-8d46524804cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:32.326594 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-1d6dd5e0-3e58-4f39-bbd4-9603aca82ba2 None None] MFA Rules not processed for user `e0d5404a4851455d9b82a9f21a641625`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:30:32.327904 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-1d6dd5e0-3e58-4f39-bbd4-9603aca82ba2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:32.389053 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a689667b-9a50-4177-bb88-8d46524804cd admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:32.401228 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1019/2024] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:30:32 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:30:32.404864 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6dd5e0-3e58-4f39-bbd4-9603aca82ba2 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:32.406372 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-1d6dd5e0-3e58-4f39-bbd4-9603aca82ba2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:32.406726 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40d92f3-229f-4b91-a666-8d5e71239767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:32.407681 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40d92f3-229f-4b91-a666-8d5e71239767 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:32.408133 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40d92f3-229f-4b91-a666-8d5e71239767 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:32.408414 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40d92f3-229f-4b91-a666-8d5e71239767 admin admin] PATH_INFO: `/v3/users/fc2cb586d2eb45bb966a010bd3ee50e2` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:32.409678 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40d92f3-229f-4b91-a666-8d5e71239767 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc2cb586d2eb45bb966a010bd3ee50e2)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:32.411640 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40d92f3-229f-4b91-a666-8d5e71239767 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:32.422226 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6dd5e0-3e58-4f39-bbd4-9603aca82ba2 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:32.431669 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a051ac05-623c-4934-bc98-832ae6221053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:32.433716 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a051ac05-623c-4934-bc98-832ae6221053 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:32.433716 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a051ac05-623c-4934-bc98-832ae6221053 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:32.433716 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a051ac05-623c-4934-bc98-832ae6221053 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:32.433716 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a051ac05-623c-4934-bc98-832ae6221053 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:32.436359 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a051ac05-623c-4934-bc98-832ae6221053 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:32.551595 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1020/2025] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 17 19:30:32 2022] POST /identity/v3/users => generated 289 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:30:32.551595 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a051ac05-623c-4934-bc98-832ae6221053 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:32.554051 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a051ac05-623c-4934-bc98-832ae6221053 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:32.566741 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a051ac05-623c-4934-bc98-832ae6221053 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:32.571506 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9055b287-3294-4919-89e6-9c0d26341382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:32.572181 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9055b287-3294-4919-89e6-9c0d26341382 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:32.572482 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9055b287-3294-4919-89e6-9c0d26341382 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:32.572739 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9055b287-3294-4919-89e6-9c0d26341382 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:32.573479 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9055b287-3294-4919-89e6-9c0d26341382 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:32.576246 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9055b287-3294-4919-89e6-9c0d26341382 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:32.680378 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-c40d92f3-229f-4b91-a666-8d5e71239767 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc2cb586d2eb45bb966a010bd3ee50e2'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:32.693636 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-c40d92f3-229f-4b91-a666-8d5e71239767 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc2cb586d2eb45bb966a010bd3ee50e2'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:32.707021 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-c40d92f3-229f-4b91-a666-8d5e71239767 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc2cb586d2eb45bb966a010bd3ee50e2'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:32.708022 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-c40d92f3-229f-4b91-a666-8d5e71239767 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc2cb586d2eb45bb966a010bd3ee50e2'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:32.708782 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-c40d92f3-229f-4b91-a666-8d5e71239767 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc2cb586d2eb45bb966a010bd3ee50e2'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:32.711030 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1021/2026] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 17 19:30:32 2022] POST /identity/v3/projects => generated 371 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:30:32.711904 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9055b287-3294-4919-89e6-9c0d26341382 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:32.717250 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-9055b287-3294-4919-89e6-9c0d26341382 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:32.783143 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9055b287-3294-4919-89e6-9c0d26341382 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:32.788052 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223d38fc-230a-40cd-a65e-f2d1a019ba81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:32.788724 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223d38fc-230a-40cd-a65e-f2d1a019ba81 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:32.789005 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223d38fc-230a-40cd-a65e-f2d1a019ba81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:32.789278 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223d38fc-230a-40cd-a65e-f2d1a019ba81 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:32.790000 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223d38fc-230a-40cd-a65e-f2d1a019ba81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:32.791660 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223d38fc-230a-40cd-a65e-f2d1a019ba81 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:32.802397 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1022/2027] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:30:32 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:30:32.803382 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223d38fc-230a-40cd-a65e-f2d1a019ba81 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:32.805089 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-223d38fc-230a-40cd-a65e-f2d1a019ba81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:32.812999 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1006/2028] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 17 19:30:32 2022] DELETE /identity/v3/users/fc2cb586d2eb45bb966a010bd3ee50e2 => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:30:32.813825 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40d92f3-229f-4b91-a666-8d5e71239767 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:32.819245 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-c40d92f3-229f-4b91-a666-8d5e71239767 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:32.819378 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223d38fc-230a-40cd-a65e-f2d1a019ba81 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:32.908328 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40d92f3-229f-4b91-a666-8d5e71239767 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:32.915608 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362923f6-0b85-4cc6-bf53-e4e39d735126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:32.919318 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362923f6-0b85-4cc6-bf53-e4e39d735126 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:32.919843 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362923f6-0b85-4cc6-bf53-e4e39d735126 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:32.920035 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362923f6-0b85-4cc6-bf53-e4e39d735126 admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:32.920777 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362923f6-0b85-4cc6-bf53-e4e39d735126 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:32.922857 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362923f6-0b85-4cc6-bf53-e4e39d735126 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:32.938754 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff601e68-fc6d-4dbc-8e26-e2319500c356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:32.942307 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff601e68-fc6d-4dbc-8e26-e2319500c356 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:32.942307 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff601e68-fc6d-4dbc-8e26-e2319500c356 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:32.942307 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff601e68-fc6d-4dbc-8e26-e2319500c356 admin admin] PATH_INFO: `/v3/users/df22e0d8a20444debd00e3cde8fc98ce` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:32.942307 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff601e68-fc6d-4dbc-8e26-e2319500c356 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df22e0d8a20444debd00e3cde8fc98ce)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:32.947521 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff601e68-fc6d-4dbc-8e26-e2319500c356 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:33.092740 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1007/2029] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 17 19:30:32 2022] POST /identity/v3/users => generated 292 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:30:33.093197 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362923f6-0b85-4cc6-bf53-e4e39d735126 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:33.118468 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362923f6-0b85-4cc6-bf53-e4e39d735126 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:33.128484 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee88706-09d1-4930-a19a-cce1851a75a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:33.129820 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee88706-09d1-4930-a19a-cce1851a75a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:33.130303 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee88706-09d1-4930-a19a-cce1851a75a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:33.130912 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee88706-09d1-4930-a19a-cce1851a75a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:33.131897 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee88706-09d1-4930-a19a-cce1851a75a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:33.133881 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee88706-09d1-4930-a19a-cce1851a75a2 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:33.147618 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1008/2030] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:30:33 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:30:33.148801 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee88706-09d1-4930-a19a-cce1851a75a2 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:33.160962 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee88706-09d1-4930-a19a-cce1851a75a2 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:33.174861 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef18935-afe1-485c-a6bd-3c5af2e33306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:33.175636 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef18935-afe1-485c-a6bd-3c5af2e33306 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:33.176002 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef18935-afe1-485c-a6bd-3c5af2e33306 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:33.176185 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef18935-afe1-485c-a6bd-3c5af2e33306 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:33.176974 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef18935-afe1-485c-a6bd-3c5af2e33306 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:33.178802 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef18935-afe1-485c-a6bd-3c5af2e33306 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:33.179897 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ff601e68-fc6d-4dbc-8e26-e2319500c356 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df22e0d8a20444debd00e3cde8fc98ce'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:33.182672 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ff601e68-fc6d-4dbc-8e26-e2319500c356 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df22e0d8a20444debd00e3cde8fc98ce'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:33.193996 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1009/2031] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:30:33 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:30:33.198099 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef18935-afe1-485c-a6bd-3c5af2e33306 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:33.200872 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-cef18935-afe1-485c-a6bd-3c5af2e33306 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:33.205050 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ff601e68-fc6d-4dbc-8e26-e2319500c356 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df22e0d8a20444debd00e3cde8fc98ce'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:33.218879 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef18935-afe1-485c-a6bd-3c5af2e33306 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:33.224756 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34350478-733d-4253-a3da-176c7f0e09e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:33.225644 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34350478-733d-4253-a3da-176c7f0e09e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:33.226026 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34350478-733d-4253-a3da-176c7f0e09e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:33.226285 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34350478-733d-4253-a3da-176c7f0e09e9 admin admin] PATH_INFO: `/v3/projects/f00cf662443f43ef951dd1aab8f35d7d/users/19becca57a494cc0bc8fc68bf9ad8c10/roles/114e6a16e63447d1924707ace5aace25` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:33.242939 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34350478-733d-4253-a3da-176c7f0e09e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f00cf662443f43ef951dd1aab8f35d7d, user_id=19becca57a494cc0bc8fc68bf9ad8c10, role_id=114e6a16e63447d1924707ace5aace25)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:33.245991 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34350478-733d-4253-a3da-176c7f0e09e9 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:33.284954 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ff601e68-fc6d-4dbc-8e26-e2319500c356 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df22e0d8a20444debd00e3cde8fc98ce'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:33.292570 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ff601e68-fc6d-4dbc-8e26-e2319500c356 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df22e0d8a20444debd00e3cde8fc98ce'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:33.298693 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1023/2032] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 17 19:30:32 2022] DELETE /identity/v3/users/df22e0d8a20444debd00e3cde8fc98ce => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:30:33.300111 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff601e68-fc6d-4dbc-8e26-e2319500c356 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:33.301908 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-ff601e68-fc6d-4dbc-8e26-e2319500c356 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:33.371007 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff601e68-fc6d-4dbc-8e26-e2319500c356 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:33.380437 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1010/2033] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:30:33 2022] PUT /identity/v3/projects/f00cf662443f43ef951dd1aab8f35d7d/users/19becca57a494cc0bc8fc68bf9ad8c10/roles/114e6a16e63447d1924707ace5aace25 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:30:33.381817 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3de5672-94d3-45b7-8d00-bdab2cbd881e None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:33.389337 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3de5672-94d3-45b7-8d00-bdab2cbd881e None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:33.389911 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3de5672-94d3-45b7-8d00-bdab2cbd881e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:33.413939 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-e3de5672-94d3-45b7-8d00-bdab2cbd881e None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:30:33.415292 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-e3de5672-94d3-45b7-8d00-bdab2cbd881e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:33.471552 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538b9546-dcb4-4ee7-becd-562c4c30ba85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:33.472840 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538b9546-dcb4-4ee7-becd-562c4c30ba85 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:33.473265 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538b9546-dcb4-4ee7-becd-562c4c30ba85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:33.473674 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538b9546-dcb4-4ee7-becd-562c4c30ba85 admin admin] PATH_INFO: `/v3/projects/21af0c27992c45b7be22b686674afef9/users/b1d22d948b1e4e10beafaeb652ce4ad9/roles/60af5088793e4379b88821a86552a344` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:33.488863 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1011/2034] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:30:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:30:33.489719 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3de5672-94d3-45b7-8d00-bdab2cbd881e None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:33.491005 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-e3de5672-94d3-45b7-8d00-bdab2cbd881e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:33.496888 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538b9546-dcb4-4ee7-becd-562c4c30ba85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21af0c27992c45b7be22b686674afef9, user_id=b1d22d948b1e4e10beafaeb652ce4ad9, role_id=60af5088793e4379b88821a86552a344)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:33.500264 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538b9546-dcb4-4ee7-becd-562c4c30ba85 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:33.507161 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3de5672-94d3-45b7-8d00-bdab2cbd881e None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:33.514062 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76689cdb-97af-47fb-8006-1381314c5b7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:33.514927 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76689cdb-97af-47fb-8006-1381314c5b7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:33.515236 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76689cdb-97af-47fb-8006-1381314c5b7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:33.515589 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76689cdb-97af-47fb-8006-1381314c5b7e admin admin] PATH_INFO: `/v3/projects/4ac95786705c46e0b49273629304d0d4` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:33.521705 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76689cdb-97af-47fb-8006-1381314c5b7e admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ac95786705c46e0b49273629304d0d4)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:33.521705 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76689cdb-97af-47fb-8006-1381314c5b7e admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:33.588877 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1024/2035] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:30:33 2022] PUT /identity/v3/projects/21af0c27992c45b7be22b686674afef9/users/b1d22d948b1e4e10beafaeb652ce4ad9/roles/60af5088793e4379b88821a86552a344 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:30:33.590197 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7d0dcf-3e0f-4d55-8cbd-8f6ef57e08f6 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:33.590479 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7d0dcf-3e0f-4d55-8cbd-8f6ef57e08f6 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:33.590745 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7d0dcf-3e0f-4d55-8cbd-8f6ef57e08f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:33.624743 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-0a7d0dcf-3e0f-4d55-8cbd-8f6ef57e08f6 None None] MFA Rules not processed for user `19becca57a494cc0bc8fc68bf9ad8c10`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:30:33.626115 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-0a7d0dcf-3e0f-4d55-8cbd-8f6ef57e08f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:33.637137 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-76689cdb-97af-47fb-8006-1381314c5b7e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ac95786705c46e0b49273629304d0d4', 'request_id': 'req-76689cdb-97af-47fb-8006-1381314c5b7e'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:33.645577 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-76689cdb-97af-47fb-8006-1381314c5b7e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ac95786705c46e0b49273629304d0d4', 'request_id': 'req-76689cdb-97af-47fb-8006-1381314c5b7e'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:33.652020 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-76689cdb-97af-47fb-8006-1381314c5b7e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ac95786705c46e0b49273629304d0d4', 'request_id': 'req-76689cdb-97af-47fb-8006-1381314c5b7e'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:33.701638 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1025/2036] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:30:33 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:30:33.702494 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7d0dcf-3e0f-4d55-8cbd-8f6ef57e08f6 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:33.703736 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-0a7d0dcf-3e0f-4d55-8cbd-8f6ef57e08f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:33.766944 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7d0dcf-3e0f-4d55-8cbd-8f6ef57e08f6 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:33.768255 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d4ba67-dbd8-4172-848b-7b72fd03e4aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:33.768984 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d4ba67-dbd8-4172-848b-7b72fd03e4aa service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:33.769292 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d4ba67-dbd8-4172-848b-7b72fd03e4aa service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:33.769507 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d4ba67-dbd8-4172-848b-7b72fd03e4aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:33.819585 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-76689cdb-97af-47fb-8006-1381314c5b7e admin admin] The token cache is being invalidate because project 4ac95786705c46e0b49273629304d0d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 17 19:30:33.820345 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-76689cdb-97af-47fb-8006-1381314c5b7e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:33.821087 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-76689cdb-97af-47fb-8006-1381314c5b7e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:33.822935 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1012/2037] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 17 19:30:33 2022] DELETE /identity/v3/projects/4ac95786705c46e0b49273629304d0d4 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:30:33.828039 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76689cdb-97af-47fb-8006-1381314c5b7e admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:33.829681 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-76689cdb-97af-47fb-8006-1381314c5b7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:33.836888 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d4ba67-dbd8-4172-848b-7b72fd03e4aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:33.838352 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d4ba67-dbd8-4172-848b-7b72fd03e4aa service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:33.846269 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76689cdb-97af-47fb-8006-1381314c5b7e admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:33.850147 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1026/2038] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:30:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:30:33.850972 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d4ba67-dbd8-4172-848b-7b72fd03e4aa service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:33.852363 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-92d4ba67-dbd8-4172-848b-7b72fd03e4aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:33.853075 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1867ac9a-845f-4bb7-a636-ae5c4052c484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:33.854352 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1867ac9a-845f-4bb7-a636-ae5c4052c484 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:33.857801 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1867ac9a-845f-4bb7-a636-ae5c4052c484 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:33.857801 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1867ac9a-845f-4bb7-a636-ae5c4052c484 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:33.857801 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1867ac9a-845f-4bb7-a636-ae5c4052c484 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:33.859292 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1867ac9a-845f-4bb7-a636-ae5c4052c484 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:33.875543 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1013/2039] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:30:33 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:30:33.875543 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23302961-4081-40ed-be6b-f676bbde2572 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:33.875543 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23302961-4081-40ed-be6b-f676bbde2572 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:33.875543 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23302961-4081-40ed-be6b-f676bbde2572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:33.904284 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-23302961-4081-40ed-be6b-f676bbde2572 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:30:33.905580 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-23302961-4081-40ed-be6b-f676bbde2572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:33.919609 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1014/2040] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:30:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:30:33.919609 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23302961-4081-40ed-be6b-f676bbde2572 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:33.927583 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23302961-4081-40ed-be6b-f676bbde2572 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:33.932881 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fe95ad-685a-4a4b-b3df-2ecbcf5e033e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:33.934938 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fe95ad-685a-4a4b-b3df-2ecbcf5e033e admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:33.934938 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fe95ad-685a-4a4b-b3df-2ecbcf5e033e admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:33.935247 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fe95ad-685a-4a4b-b3df-2ecbcf5e033e admin admin] PATH_INFO: `/v3/projects/21af0c27992c45b7be22b686674afef9/users/b1d22d948b1e4e10beafaeb652ce4ad9/roles/60af5088793e4379b88821a86552a344` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:33.939607 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fe95ad-685a-4a4b-b3df-2ecbcf5e033e admin admin] RBAC: Authorizing `identity:create_grant(project_id=21af0c27992c45b7be22b686674afef9, user_id=b1d22d948b1e4e10beafaeb652ce4ad9, role_id=60af5088793e4379b88821a86552a344)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:33.942702 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fe95ad-685a-4a4b-b3df-2ecbcf5e033e admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:33.945049 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d4ba67-dbd8-4172-848b-7b72fd03e4aa service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:33.954362 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873b150e-3bb2-4c83-829d-d10c123a5b9a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd05409b38ae346458abfe41ebda1c59e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:33.957966 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873b150e-3bb2-4c83-829d-d10c123a5b9a service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:33.957966 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873b150e-3bb2-4c83-829d-d10c123a5b9a service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:33.957966 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873b150e-3bb2-4c83-829d-d10c123a5b9a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:33.959017 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1015/2041] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:30:33 2022] PUT /identity/v3/projects/21af0c27992c45b7be22b686674afef9/users/b1d22d948b1e4e10beafaeb652ce4ad9/roles/60af5088793e4379b88821a86552a344 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:30:33.959800 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-873b150e-3bb2-4c83-829d-d10c123a5b9a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:33.963855 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2249979-74fb-4726-997c-83d808342289 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:33.964099 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2249979-74fb-4726-997c-83d808342289 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:33.964429 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2249979-74fb-4726-997c-83d808342289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:33.974665 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873b150e-3bb2-4c83-829d-d10c123a5b9a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:33.976791 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873b150e-3bb2-4c83-829d-d10c123a5b9a service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:33.987965 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1027/2042] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:30:33 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:30:33.988287 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-e2249979-74fb-4726-997c-83d808342289 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:30:33.989523 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-e2249979-74fb-4726-997c-83d808342289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:34.067209 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1016/2043] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:30:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:30:34.078813 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873b150e-3bb2-4c83-829d-d10c123a5b9a service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:34.090753 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873b150e-3bb2-4c83-829d-d10c123a5b9a service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:34.096951 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a060f4e-768d-41dd-a41a-aac471dd5e33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:34.097711 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a060f4e-768d-41dd-a41a-aac471dd5e33 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:34.098081 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a060f4e-768d-41dd-a41a-aac471dd5e33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:34.098507 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a060f4e-768d-41dd-a41a-aac471dd5e33 admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:34.099165 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a060f4e-768d-41dd-a41a-aac471dd5e33 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:30:34.099811 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a060f4e-768d-41dd-a41a-aac471dd5e33 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:34.101618 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a060f4e-768d-41dd-a41a-aac471dd5e33 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:34.121006 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2249979-74fb-4726-997c-83d808342289 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:34.127082 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1028/2044] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 17 19:30:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:30:34.134419 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2249979-74fb-4726-997c-83d808342289 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:34.143772 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b45faa8-6521-4a91-be8f-8cb1b35f8ca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:34.145380 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b45faa8-6521-4a91-be8f-8cb1b35f8ca2 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:34.145730 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b45faa8-6521-4a91-be8f-8cb1b35f8ca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:34.146071 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b45faa8-6521-4a91-be8f-8cb1b35f8ca2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:34.147004 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b45faa8-6521-4a91-be8f-8cb1b35f8ca2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:34.149498 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b45faa8-6521-4a91-be8f-8cb1b35f8ca2 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:34.249530 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1017/2045] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 17 19:30:34 2022] POST /identity/v3/projects => generated 371 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:30:34.253828 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a060f4e-768d-41dd-a41a-aac471dd5e33 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:34.269108 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a060f4e-768d-41dd-a41a-aac471dd5e33 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:34.386029 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60c836a-46eb-4eba-82fa-1085409a2c6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:34.387782 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60c836a-46eb-4eba-82fa-1085409a2c6b admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:34.388125 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60c836a-46eb-4eba-82fa-1085409a2c6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:34.388472 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60c836a-46eb-4eba-82fa-1085409a2c6b admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:34.389910 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60c836a-46eb-4eba-82fa-1085409a2c6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:34.392743 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60c836a-46eb-4eba-82fa-1085409a2c6b admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:34.508226 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1029/2046] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 17 19:30:34 2022] POST /identity/v3/users => generated 286 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:30:34.508404 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b45faa8-6521-4a91-be8f-8cb1b35f8ca2 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:34.516664 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b45faa8-6521-4a91-be8f-8cb1b35f8ca2 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:34.521476 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3ae5f9-2ecf-4717-8671-c3f3162578f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:34.522141 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3ae5f9-2ecf-4717-8671-c3f3162578f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:34.522423 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3ae5f9-2ecf-4717-8671-c3f3162578f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:34.522690 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3ae5f9-2ecf-4717-8671-c3f3162578f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:34.523411 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3ae5f9-2ecf-4717-8671-c3f3162578f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:34.525093 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3ae5f9-2ecf-4717-8671-c3f3162578f3 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:34.551752 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60c836a-46eb-4eba-82fa-1085409a2c6b admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:34.552939 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1018/2047] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:30:34 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:30:34.556971 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3ae5f9-2ecf-4717-8671-c3f3162578f3 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:34.557581 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-e60c836a-46eb-4eba-82fa-1085409a2c6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:34.577366 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3ae5f9-2ecf-4717-8671-c3f3162578f3 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:34.583515 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60c836a-46eb-4eba-82fa-1085409a2c6b admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:34.584158 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d89eac2-4991-45dc-96e5-ab4466e09e25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:34.584813 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d89eac2-4991-45dc-96e5-ab4466e09e25 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:34.585089 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d89eac2-4991-45dc-96e5-ab4466e09e25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:34.585360 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d89eac2-4991-45dc-96e5-ab4466e09e25 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:34.586081 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d89eac2-4991-45dc-96e5-ab4466e09e25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:34.587923 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d89eac2-4991-45dc-96e5-ab4466e09e25 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:34.593416 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2ded78-2d59-4f13-a535-3d84a45bd473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:34.594500 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2ded78-2d59-4f13-a535-3d84a45bd473 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:34.594922 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2ded78-2d59-4f13-a535-3d84a45bd473 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:34.595780 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2ded78-2d59-4f13-a535-3d84a45bd473 admin admin] PATH_INFO: `/v3/projects/97d6875a858449e7b8062238704a4769` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:34.599112 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2ded78-2d59-4f13-a535-3d84a45bd473 admin admin] RBAC: Authorizing `identity:delete_project(project_id=97d6875a858449e7b8062238704a4769)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:34.600974 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1019/2048] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:30:34 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:30:34.601915 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2ded78-2d59-4f13-a535-3d84a45bd473 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:34.604934 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d89eac2-4991-45dc-96e5-ab4466e09e25 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:34.616451 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d89eac2-4991-45dc-96e5-ab4466e09e25 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:34.623559 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879c180a-7acb-4537-9cba-5834e007536b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:34.627008 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879c180a-7acb-4537-9cba-5834e007536b admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:34.627008 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879c180a-7acb-4537-9cba-5834e007536b admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:34.627008 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879c180a-7acb-4537-9cba-5834e007536b admin admin] PATH_INFO: `/v3/projects/09e7fe1c9d0d4693bbcdfa98e67b7795/users/0fd89b60f636452da1fb967bb546a70a/roles/114e6a16e63447d1924707ace5aace25` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:34.644648 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879c180a-7acb-4537-9cba-5834e007536b admin admin] RBAC: Authorizing `identity:create_grant(project_id=09e7fe1c9d0d4693bbcdfa98e67b7795, user_id=0fd89b60f636452da1fb967bb546a70a, role_id=114e6a16e63447d1924707ace5aace25)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:34.647604 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879c180a-7acb-4537-9cba-5834e007536b admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:34.785228 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1020/2049] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:30:34 2022] PUT /identity/v3/projects/09e7fe1c9d0d4693bbcdfa98e67b7795/users/0fd89b60f636452da1fb967bb546a70a/roles/114e6a16e63447d1924707ace5aace25 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 17 19:30:34.797041 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafd3a56-b54a-4584-a9f7-5e4ab16191a8 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:34.797360 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafd3a56-b54a-4584-a9f7-5e4ab16191a8 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:34.797818 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafd3a56-b54a-4584-a9f7-5e4ab16191a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:34.800312 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-3a2ded78-2d59-4f13-a535-3d84a45bd473 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97d6875a858449e7b8062238704a4769', 'request_id': 'req-3a2ded78-2d59-4f13-a535-3d84a45bd473'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:34.807940 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-3a2ded78-2d59-4f13-a535-3d84a45bd473 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97d6875a858449e7b8062238704a4769', 'request_id': 'req-3a2ded78-2d59-4f13-a535-3d84a45bd473'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:34.838365 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-dafd3a56-b54a-4584-a9f7-5e4ab16191a8 None None] MFA Rules not processed for user `0fd89b60f636452da1fb967bb546a70a`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:30:34.839780 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-dafd3a56-b54a-4584-a9f7-5e4ab16191a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:34.895643 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-3a2ded78-2d59-4f13-a535-3d84a45bd473 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97d6875a858449e7b8062238704a4769', 'request_id': 'req-3a2ded78-2d59-4f13-a535-3d84a45bd473'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:34.915211 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-3a2ded78-2d59-4f13-a535-3d84a45bd473 admin admin] The token cache is being invalidate because project 97d6875a858449e7b8062238704a4769 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 17 19:30:34.916340 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-3a2ded78-2d59-4f13-a535-3d84a45bd473 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:34.917013 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-3a2ded78-2d59-4f13-a535-3d84a45bd473 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:34.922403 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1030/2050] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 17 19:30:34 2022] DELETE /identity/v3/projects/97d6875a858449e7b8062238704a4769 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:30:34.925788 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2ded78-2d59-4f13-a535-3d84a45bd473 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:34.928069 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-3a2ded78-2d59-4f13-a535-3d84a45bd473 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:34.983401 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1021/2051] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:30:34 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:30:35.000828 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafd3a56-b54a-4584-a9f7-5e4ab16191a8 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:35.002333 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-dafd3a56-b54a-4584-a9f7-5e4ab16191a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:35.054132 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2ded78-2d59-4f13-a535-3d84a45bd473 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:35.095883 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafd3a56-b54a-4584-a9f7-5e4ab16191a8 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:35.101806 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8127ef8-0c9f-4a16-aaef-f800bea10546 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd05409b38ae346458abfe41ebda1c59e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:35.102959 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8127ef8-0c9f-4a16-aaef-f800bea10546 service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:35.103244 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8127ef8-0c9f-4a16-aaef-f800bea10546 service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:35.103570 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8127ef8-0c9f-4a16-aaef-f800bea10546 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:35.119530 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8127ef8-0c9f-4a16-aaef-f800bea10546 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:35.120327 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8127ef8-0c9f-4a16-aaef-f800bea10546 service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:35.131036 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1022/2052] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:30:34 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:30:35.132713 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1ecbe6-c539-4a0b-9b0a-2d27aee7f786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:35.133942 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1ecbe6-c539-4a0b-9b0a-2d27aee7f786 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:35.134574 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1ecbe6-c539-4a0b-9b0a-2d27aee7f786 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:35.135242 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1ecbe6-c539-4a0b-9b0a-2d27aee7f786 admin admin] PATH_INFO: `/v3/users/095dbdc7dbc942a28bde981ce73684af` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:35.137656 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1ecbe6-c539-4a0b-9b0a-2d27aee7f786 admin admin] RBAC: Authorizing `identity:delete_user(user_id=095dbdc7dbc942a28bde981ce73684af)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:35.139831 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1ecbe6-c539-4a0b-9b0a-2d27aee7f786 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:35.161516 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8127ef8-0c9f-4a16-aaef-f800bea10546 service cinder] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:35.163093 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-e8127ef8-0c9f-4a16-aaef-f800bea10546 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:35.180000 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8127ef8-0c9f-4a16-aaef-f800bea10546 service cinder] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:35.184881 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33193601-38cc-485e-8e16-4352cb21ec5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:35.185742 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33193601-38cc-485e-8e16-4352cb21ec5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:35.186022 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33193601-38cc-485e-8e16-4352cb21ec5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:35.186297 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33193601-38cc-485e-8e16-4352cb21ec5f admin admin] PATH_INFO: `/v3/users/0fd89b60f636452da1fb967bb546a70a` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:35.187610 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33193601-38cc-485e-8e16-4352cb21ec5f admin admin] RBAC: Authorizing `identity:delete_user(user_id=0fd89b60f636452da1fb967bb546a70a)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:35.189546 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33193601-38cc-485e-8e16-4352cb21ec5f admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:35.438520 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-33193601-38cc-485e-8e16-4352cb21ec5f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0fd89b60f636452da1fb967bb546a70a'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:35.442423 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0f1ecbe6-c539-4a0b-9b0a-2d27aee7f786 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '095dbdc7dbc942a28bde981ce73684af'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:35.443561 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0f1ecbe6-c539-4a0b-9b0a-2d27aee7f786 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '095dbdc7dbc942a28bde981ce73684af'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:35.451808 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-33193601-38cc-485e-8e16-4352cb21ec5f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0fd89b60f636452da1fb967bb546a70a'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:35.465454 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0f1ecbe6-c539-4a0b-9b0a-2d27aee7f786 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '095dbdc7dbc942a28bde981ce73684af'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:35.477477 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-33193601-38cc-485e-8e16-4352cb21ec5f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0fd89b60f636452da1fb967bb546a70a'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:35.478827 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-33193601-38cc-485e-8e16-4352cb21ec5f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0fd89b60f636452da1fb967bb546a70a'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:35.479271 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-33193601-38cc-485e-8e16-4352cb21ec5f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0fd89b60f636452da1fb967bb546a70a'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:35.577668 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0f1ecbe6-c539-4a0b-9b0a-2d27aee7f786 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '095dbdc7dbc942a28bde981ce73684af'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:35.588577 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0f1ecbe6-c539-4a0b-9b0a-2d27aee7f786 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '095dbdc7dbc942a28bde981ce73684af'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:35.607317 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1031/2053] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 17 19:30:34 2022] DELETE /identity/v3/users/095dbdc7dbc942a28bde981ce73684af => generated 0 bytes in 682 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:30:35.631757 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b1945a-5e4f-420f-8909-3ea9aa215004 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:35.633967 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b1945a-5e4f-420f-8909-3ea9aa215004 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:35.638433 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b1945a-5e4f-420f-8909-3ea9aa215004 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:35.675191 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1023/2054] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 17 19:30:35 2022] DELETE /identity/v3/users/0fd89b60f636452da1fb967bb546a70a => generated 0 bytes in 514 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 17 19:30:35.678790 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-13b1945a-5e4f-420f-8909-3ea9aa215004 None None] MFA Rules not processed for user `b1d22d948b1e4e10beafaeb652ce4ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:30:35.679792 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3475b730-c788-43f7-8187-1d648921bb2f None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:35.680237 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3475b730-c788-43f7-8187-1d648921bb2f None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:35.680508 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3475b730-c788-43f7-8187-1d648921bb2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:35.680605 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-13b1945a-5e4f-420f-8909-3ea9aa215004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:35.718099 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-3475b730-c788-43f7-8187-1d648921bb2f None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:30:35.720261 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-3475b730-c788-43f7-8187-1d648921bb2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:35.829472 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1032/2055] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:30:35 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:30:35.837185 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b1945a-5e4f-420f-8909-3ea9aa215004 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:35.841757 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-13b1945a-5e4f-420f-8909-3ea9aa215004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:35.857866 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1024/2056] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:30:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:30:35.870048 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3475b730-c788-43f7-8187-1d648921bb2f None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:35.872618 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-3475b730-c788-43f7-8187-1d648921bb2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:35.929644 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b1945a-5e4f-420f-8909-3ea9aa215004 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:35.934416 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b2e633-60d1-48c5-91d9-15d7cc65043a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd05409b38ae346458abfe41ebda1c59e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:35.935196 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b2e633-60d1-48c5-91d9-15d7cc65043a service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:35.935730 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b2e633-60d1-48c5-91d9-15d7cc65043a service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:35.936081 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b2e633-60d1-48c5-91d9-15d7cc65043a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:35.938261 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-85b2e633-60d1-48c5-91d9-15d7cc65043a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:36.015132 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3475b730-c788-43f7-8187-1d648921bb2f None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:36.022433 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcf9b8c-e156-46b1-8999-ac45dc0b3c64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:36.023083 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcf9b8c-e156-46b1-8999-ac45dc0b3c64 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:36.023610 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcf9b8c-e156-46b1-8999-ac45dc0b3c64 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:36.026463 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcf9b8c-e156-46b1-8999-ac45dc0b3c64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:36.031920 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b2e633-60d1-48c5-91d9-15d7cc65043a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:36.033354 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b2e633-60d1-48c5-91d9-15d7cc65043a service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:36.045385 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcf9b8c-e156-46b1-8999-ac45dc0b3c64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:36.046853 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcf9b8c-e156-46b1-8999-ac45dc0b3c64 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:36.057187 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1033/2057] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:30:35 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 17 19:30:36.077563 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1025/2058] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:30:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:30:36.485879 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b2e633-60d1-48c5-91d9-15d7cc65043a service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:36.487725 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-85b2e633-60d1-48c5-91d9-15d7cc65043a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:36.513812 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b2e633-60d1-48c5-91d9-15d7cc65043a service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:36.539403 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccdd439-d2ad-48c9-b1d1-8409ac606015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:36.540306 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccdd439-d2ad-48c9-b1d1-8409ac606015 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:36.540582 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccdd439-d2ad-48c9-b1d1-8409ac606015 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:36.541478 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccdd439-d2ad-48c9-b1d1-8409ac606015 admin admin] PATH_INFO: `/v3/projects/646656a4b9a343e490fa2a954f614ff8` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:36.544725 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccdd439-d2ad-48c9-b1d1-8409ac606015 admin admin] RBAC: Authorizing `identity:delete_project(project_id=646656a4b9a343e490fa2a954f614ff8)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:36.549939 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccdd439-d2ad-48c9-b1d1-8409ac606015 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:36.756421 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcf9b8c-e156-46b1-8999-ac45dc0b3c64 service neutron] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:36.759377 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-3fcf9b8c-e156-46b1-8999-ac45dc0b3c64 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:36.779658 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-eccdd439-d2ad-48c9-b1d1-8409ac606015 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '646656a4b9a343e490fa2a954f614ff8', 'request_id': 'req-eccdd439-d2ad-48c9-b1d1-8409ac606015'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:36.782922 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcf9b8c-e156-46b1-8999-ac45dc0b3c64 service neutron] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:36.799617 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-eccdd439-d2ad-48c9-b1d1-8409ac606015 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '646656a4b9a343e490fa2a954f614ff8', 'request_id': 'req-eccdd439-d2ad-48c9-b1d1-8409ac606015'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:36.891671 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7117b79-9549-41cc-859c-f6dcc568c98e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:36.892752 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7117b79-9549-41cc-859c-f6dcc568c98e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:36.893033 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7117b79-9549-41cc-859c-f6dcc568c98e admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:36.893305 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7117b79-9549-41cc-859c-f6dcc568c98e admin admin] PATH_INFO: `/v3/users/b516f9768e66489d8c555fcb4c803e10` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:36.894608 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7117b79-9549-41cc-859c-f6dcc568c98e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b516f9768e66489d8c555fcb4c803e10)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:36.896656 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7117b79-9549-41cc-859c-f6dcc568c98e admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:36.900927 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-eccdd439-d2ad-48c9-b1d1-8409ac606015 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '646656a4b9a343e490fa2a954f614ff8', 'request_id': 'req-eccdd439-d2ad-48c9-b1d1-8409ac606015'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:36.917458 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-eccdd439-d2ad-48c9-b1d1-8409ac606015 admin admin] The token cache is being invalidate because project 646656a4b9a343e490fa2a954f614ff8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 17 19:30:36.917749 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-eccdd439-d2ad-48c9-b1d1-8409ac606015 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:36.918683 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-eccdd439-d2ad-48c9-b1d1-8409ac606015 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:36.921835 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1034/2059] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 17 19:30:36 2022] DELETE /identity/v3/projects/646656a4b9a343e490fa2a954f614ff8 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:30:36.923247 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccdd439-d2ad-48c9-b1d1-8409ac606015 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:36.925198 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-eccdd439-d2ad-48c9-b1d1-8409ac606015 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:36.948230 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccdd439-d2ad-48c9-b1d1-8409ac606015 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:36.956255 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8725de2b-7d4d-4b53-bcd2-10ec0b46989d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:36.957123 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8725de2b-7d4d-4b53-bcd2-10ec0b46989d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:36.957670 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8725de2b-7d4d-4b53-bcd2-10ec0b46989d admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:36.957946 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8725de2b-7d4d-4b53-bcd2-10ec0b46989d admin admin] PATH_INFO: `/v3/projects/09e7fe1c9d0d4693bbcdfa98e67b7795` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:36.959344 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8725de2b-7d4d-4b53-bcd2-10ec0b46989d admin admin] RBAC: Authorizing `identity:delete_project(project_id=09e7fe1c9d0d4693bbcdfa98e67b7795)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:36.961330 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8725de2b-7d4d-4b53-bcd2-10ec0b46989d admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:37.101251 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-c7117b79-9549-41cc-859c-f6dcc568c98e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b516f9768e66489d8c555fcb4c803e10'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:37.108487 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-c7117b79-9549-41cc-859c-f6dcc568c98e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b516f9768e66489d8c555fcb4c803e10'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:37.109685 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-8725de2b-7d4d-4b53-bcd2-10ec0b46989d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09e7fe1c9d0d4693bbcdfa98e67b7795', 'request_id': 'req-8725de2b-7d4d-4b53-bcd2-10ec0b46989d'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:37.117625 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-8725de2b-7d4d-4b53-bcd2-10ec0b46989d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09e7fe1c9d0d4693bbcdfa98e67b7795', 'request_id': 'req-8725de2b-7d4d-4b53-bcd2-10ec0b46989d'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:37.122211 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-c7117b79-9549-41cc-859c-f6dcc568c98e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b516f9768e66489d8c555fcb4c803e10'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:37.122939 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-c7117b79-9549-41cc-859c-f6dcc568c98e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b516f9768e66489d8c555fcb4c803e10'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:37.123629 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-c7117b79-9549-41cc-859c-f6dcc568c98e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b516f9768e66489d8c555fcb4c803e10'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:37.219271 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-8725de2b-7d4d-4b53-bcd2-10ec0b46989d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09e7fe1c9d0d4693bbcdfa98e67b7795', 'request_id': 'req-8725de2b-7d4d-4b53-bcd2-10ec0b46989d'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:37.231549 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-8725de2b-7d4d-4b53-bcd2-10ec0b46989d admin admin] The token cache is being invalidate because project 09e7fe1c9d0d4693bbcdfa98e67b7795 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 17 19:30:37.231817 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-8725de2b-7d4d-4b53-bcd2-10ec0b46989d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:37.234011 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-8725de2b-7d4d-4b53-bcd2-10ec0b46989d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:37.235965 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1035/2060] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 17 19:30:36 2022] DELETE /identity/v3/projects/09e7fe1c9d0d4693bbcdfa98e67b7795 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:30:37.236852 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8725de2b-7d4d-4b53-bcd2-10ec0b46989d admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:37.238246 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-8725de2b-7d4d-4b53-bcd2-10ec0b46989d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:37.304162 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8725de2b-7d4d-4b53-bcd2-10ec0b46989d admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:37.308377 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1026/2061] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 17 19:30:36 2022] DELETE /identity/v3/users/b516f9768e66489d8c555fcb4c803e10 => generated 0 bytes in 553 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:30:37.309723 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fd0713-6978-4917-adce-ab5b9a269fcb None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:37.310001 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fd0713-6978-4917-adce-ab5b9a269fcb None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:37.310269 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fd0713-6978-4917-adce-ab5b9a269fcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:37.312202 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f881b31-abbd-41f5-b139-e8f6f92c270d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:37.313268 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f881b31-abbd-41f5-b139-e8f6f92c270d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:37.313549 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f881b31-abbd-41f5-b139-e8f6f92c270d admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:37.313824 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f881b31-abbd-41f5-b139-e8f6f92c270d admin admin] PATH_INFO: `/v3/users/87e160fa85924c1e86880e44a41d35c5` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:37.315017 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f881b31-abbd-41f5-b139-e8f6f92c270d admin admin] RBAC: Authorizing `identity:delete_user(user_id=87e160fa85924c1e86880e44a41d35c5)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:37.317036 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f881b31-abbd-41f5-b139-e8f6f92c270d admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:37.329883 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-b0fd0713-6978-4917-adce-ab5b9a269fcb None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:30:37.332367 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-b0fd0713-6978-4917-adce-ab5b9a269fcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:37.449680 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1027/2062] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:30:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:30:37.449680 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2241fe2f-ca49-4994-a7a2-80b31ca953c9 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:37.449680 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2241fe2f-ca49-4994-a7a2-80b31ca953c9 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:37.449680 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2241fe2f-ca49-4994-a7a2-80b31ca953c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:37.449680 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-2241fe2f-ca49-4994-a7a2-80b31ca953c9 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:30:37.449680 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-2241fe2f-ca49-4994-a7a2-80b31ca953c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:37.450607 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1028/2063] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:30:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:30:37.457425 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953f90d8-a58c-4dcb-8706-5d40a364d18f None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:37.457717 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953f90d8-a58c-4dcb-8706-5d40a364d18f None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:37.458048 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953f90d8-a58c-4dcb-8706-5d40a364d18f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:37.478564 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-953f90d8-a58c-4dcb-8706-5d40a364d18f None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:30:37.479831 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-953f90d8-a58c-4dcb-8706-5d40a364d18f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:37.487761 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1029/2064] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:30:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:30:37.489284 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953f90d8-a58c-4dcb-8706-5d40a364d18f None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:37.516593 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-953f90d8-a58c-4dcb-8706-5d40a364d18f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:37.551712 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953f90d8-a58c-4dcb-8706-5d40a364d18f None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:37.567103 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09065be1-d45f-4339-9913-a36b021c3efa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:37.567103 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09065be1-d45f-4339-9913-a36b021c3efa service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:37.567103 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09065be1-d45f-4339-9913-a36b021c3efa service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:37.567103 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09065be1-d45f-4339-9913-a36b021c3efa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:37.567474 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09065be1-d45f-4339-9913-a36b021c3efa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:37.567474 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09065be1-d45f-4339-9913-a36b021c3efa service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:37.577491 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1030/2065] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:30:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:30:37.580544 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09065be1-d45f-4339-9913-a36b021c3efa service neutron] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:37.590808 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09065be1-d45f-4339-9913-a36b021c3efa service neutron] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:37.596467 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0270ef9c-3993-49e2-b7b7-6af3f78fda4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:37.597148 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0270ef9c-3993-49e2-b7b7-6af3f78fda4b admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:37.597503 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0270ef9c-3993-49e2-b7b7-6af3f78fda4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:37.597844 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0270ef9c-3993-49e2-b7b7-6af3f78fda4b admin admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:37.598654 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0270ef9c-3993-49e2-b7b7-6af3f78fda4b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:30:37.598959 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0270ef9c-3993-49e2-b7b7-6af3f78fda4b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:37.600678 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0270ef9c-3993-49e2-b7b7-6af3f78fda4b admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:37.619451 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1031/2066] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 17 19:30:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:30:37.624300 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0270ef9c-3993-49e2-b7b7-6af3f78fda4b admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:37.634005 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0270ef9c-3993-49e2-b7b7-6af3f78fda4b admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:37.639182 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b54676-53f9-45b9-b0f7-e53401380da1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:37.788885 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b54676-53f9-45b9-b0f7-e53401380da1 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:37.788885 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b54676-53f9-45b9-b0f7-e53401380da1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:37.788885 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b54676-53f9-45b9-b0f7-e53401380da1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:37.788885 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b54676-53f9-45b9-b0f7-e53401380da1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:37.788885 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b54676-53f9-45b9-b0f7-e53401380da1 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:37.826242 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0f881b31-abbd-41f5-b139-e8f6f92c270d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87e160fa85924c1e86880e44a41d35c5'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:37.826242 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0f881b31-abbd-41f5-b139-e8f6f92c270d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87e160fa85924c1e86880e44a41d35c5'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:37.832000 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0f881b31-abbd-41f5-b139-e8f6f92c270d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87e160fa85924c1e86880e44a41d35c5'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:37.986572 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1032/2067] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 17 19:30:37 2022] POST /identity/v3/projects => generated 377 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 17 19:30:37.986572 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b54676-53f9-45b9-b0f7-e53401380da1 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:37.987740 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-56b54676-53f9-45b9-b0f7-e53401380da1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:38.039476 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0f881b31-abbd-41f5-b139-e8f6f92c270d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87e160fa85924c1e86880e44a41d35c5'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:38.048720 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0f881b31-abbd-41f5-b139-e8f6f92c270d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87e160fa85924c1e86880e44a41d35c5'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:38.061484 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1036/2068] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 17 19:30:37 2022] DELETE /identity/v3/users/87e160fa85924c1e86880e44a41d35c5 => generated 0 bytes in 821 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:30:38.080427 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b54676-53f9-45b9-b0f7-e53401380da1 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:38.189501 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be1a8b4-dc2a-4d39-a3e7-aceb58f62edd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:38.190344 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be1a8b4-dc2a-4d39-a3e7-aceb58f62edd admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:38.191039 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be1a8b4-dc2a-4d39-a3e7-aceb58f62edd admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:38.191039 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be1a8b4-dc2a-4d39-a3e7-aceb58f62edd admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:38.192296 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be1a8b4-dc2a-4d39-a3e7-aceb58f62edd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:38.195206 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be1a8b4-dc2a-4d39-a3e7-aceb58f62edd admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:38.510227 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f881b31-abbd-41f5-b139-e8f6f92c270d admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:38.512239 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-0f881b31-abbd-41f5-b139-e8f6f92c270d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:38.516081 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1033/2069] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 17 19:30:37 2022] POST /identity/v3/users => generated 289 bytes in 531 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 17 19:30:38.518811 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be1a8b4-dc2a-4d39-a3e7-aceb58f62edd admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:38.539939 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be1a8b4-dc2a-4d39-a3e7-aceb58f62edd admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:38.551593 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f881b31-abbd-41f5-b139-e8f6f92c270d admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:38.556850 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08561950-fc03-4d7d-b1d0-307b430cce6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:38.558147 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08561950-fc03-4d7d-b1d0-307b430cce6b admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:38.558147 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08561950-fc03-4d7d-b1d0-307b430cce6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:38.558147 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08561950-fc03-4d7d-b1d0-307b430cce6b admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:38.559390 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08561950-fc03-4d7d-b1d0-307b430cce6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:38.559966 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73682d31-d2d5-409d-8e51-a035f03b0ddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:38.560950 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73682d31-d2d5-409d-8e51-a035f03b0ddd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:38.561182 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08561950-fc03-4d7d-b1d0-307b430cce6b admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:38.561953 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73682d31-d2d5-409d-8e51-a035f03b0ddd admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:38.562359 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73682d31-d2d5-409d-8e51-a035f03b0ddd admin admin] PATH_INFO: `/v3/projects/fd0d8bc91c7c426596a3e706f5ed7856` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:38.566307 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73682d31-d2d5-409d-8e51-a035f03b0ddd admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd0d8bc91c7c426596a3e706f5ed7856)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:38.568836 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73682d31-d2d5-409d-8e51-a035f03b0ddd admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:38.588176 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1034/2070] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:30:38 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:30:38.597825 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08561950-fc03-4d7d-b1d0-307b430cce6b admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:38.614149 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08561950-fc03-4d7d-b1d0-307b430cce6b admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:38.624928 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17bec21-a7e4-441e-a2b8-3e28f9d24d42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:38.625930 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17bec21-a7e4-441e-a2b8-3e28f9d24d42 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:38.627548 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17bec21-a7e4-441e-a2b8-3e28f9d24d42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:38.627548 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17bec21-a7e4-441e-a2b8-3e28f9d24d42 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:38.627548 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17bec21-a7e4-441e-a2b8-3e28f9d24d42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:38.629467 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17bec21-a7e4-441e-a2b8-3e28f9d24d42 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:38.643540 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1035/2071] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:30:38 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 17 19:30:38.648535 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17bec21-a7e4-441e-a2b8-3e28f9d24d42 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:38.659436 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17bec21-a7e4-441e-a2b8-3e28f9d24d42 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:38.665327 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745da456-278f-4881-9296-b0a6130f945c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:38.666251 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745da456-278f-4881-9296-b0a6130f945c admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:38.666680 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745da456-278f-4881-9296-b0a6130f945c admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:38.666959 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745da456-278f-4881-9296-b0a6130f945c admin admin] PATH_INFO: `/v3/projects/e84051f21083467db3cb5bb13d8950e0/users/9ea19110530a4a5aa030c1567ed975a3/roles/114e6a16e63447d1924707ace5aace25` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:38.694640 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745da456-278f-4881-9296-b0a6130f945c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e84051f21083467db3cb5bb13d8950e0, user_id=9ea19110530a4a5aa030c1567ed975a3, role_id=114e6a16e63447d1924707ace5aace25)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:38.695311 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745da456-278f-4881-9296-b0a6130f945c admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:38.712091 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-73682d31-d2d5-409d-8e51-a035f03b0ddd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd0d8bc91c7c426596a3e706f5ed7856', 'request_id': 'req-73682d31-d2d5-409d-8e51-a035f03b0ddd'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:38.718803 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-73682d31-d2d5-409d-8e51-a035f03b0ddd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd0d8bc91c7c426596a3e706f5ed7856', 'request_id': 'req-73682d31-d2d5-409d-8e51-a035f03b0ddd'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:38.800114 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1036/2072] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:30:38 2022] PUT /identity/v3/projects/e84051f21083467db3cb5bb13d8950e0/users/9ea19110530a4a5aa030c1567ed975a3/roles/114e6a16e63447d1924707ace5aace25 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:30:38.811216 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dca50f9-7ea1-4e78-9532-854ac82f58d1 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:38.811502 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dca50f9-7ea1-4e78-9532-854ac82f58d1 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:38.811779 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dca50f9-7ea1-4e78-9532-854ac82f58d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:38.823478 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-73682d31-d2d5-409d-8e51-a035f03b0ddd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd0d8bc91c7c426596a3e706f5ed7856', 'request_id': 'req-73682d31-d2d5-409d-8e51-a035f03b0ddd'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:38.857311 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-73682d31-d2d5-409d-8e51-a035f03b0ddd admin admin] The token cache is being invalidate because project fd0d8bc91c7c426596a3e706f5ed7856 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 17 19:30:38.857865 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-73682d31-d2d5-409d-8e51-a035f03b0ddd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:38.858304 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-73682d31-d2d5-409d-8e51-a035f03b0ddd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:38.863137 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1037/2073] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 17 19:30:38 2022] DELETE /identity/v3/projects/fd0d8bc91c7c426596a3e706f5ed7856 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 17 19:30:38.863137 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73682d31-d2d5-409d-8e51-a035f03b0ddd admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:38.876731 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-73682d31-d2d5-409d-8e51-a035f03b0ddd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:38.878890 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-0dca50f9-7ea1-4e78-9532-854ac82f58d1 None None] MFA Rules not processed for user `9ea19110530a4a5aa030c1567ed975a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:30:38.880151 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-0dca50f9-7ea1-4e78-9532-854ac82f58d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:38.948397 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73682d31-d2d5-409d-8e51-a035f03b0ddd admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:38.983129 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1037/2074] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:30:38 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 17 19:30:38.984095 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dca50f9-7ea1-4e78-9532-854ac82f58d1 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:38.985320 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-0dca50f9-7ea1-4e78-9532-854ac82f58d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:39.017040 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75037e5e-548a-4899-81b0-cafc65f08234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:39.017913 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75037e5e-548a-4899-81b0-cafc65f08234 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:39.018180 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75037e5e-548a-4899-81b0-cafc65f08234 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:39.018464 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75037e5e-548a-4899-81b0-cafc65f08234 admin admin] PATH_INFO: `/v3/projects/8cadb52387c240d09424f2b51a15a2aa` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:39.020139 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75037e5e-548a-4899-81b0-cafc65f08234 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8cadb52387c240d09424f2b51a15a2aa)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:39.022062 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75037e5e-548a-4899-81b0-cafc65f08234 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:39.065179 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dca50f9-7ea1-4e78-9532-854ac82f58d1 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:39.072325 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c87af8-0708-49a2-9af0-89578959313f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:39.073207 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c87af8-0708-49a2-9af0-89578959313f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:39.074141 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c87af8-0708-49a2-9af0-89578959313f admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:39.074589 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c87af8-0708-49a2-9af0-89578959313f admin admin] PATH_INFO: `/v3/users/07d1e046c9be49d7a889f4461ba35ca5` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:39.076302 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c87af8-0708-49a2-9af0-89578959313f admin admin] RBAC: Authorizing `identity:delete_user(user_id=07d1e046c9be49d7a889f4461ba35ca5)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:39.078481 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c87af8-0708-49a2-9af0-89578959313f admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:39.205522 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-75037e5e-548a-4899-81b0-cafc65f08234 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8cadb52387c240d09424f2b51a15a2aa', 'request_id': 'req-75037e5e-548a-4899-81b0-cafc65f08234'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:39.210932 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-75037e5e-548a-4899-81b0-cafc65f08234 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8cadb52387c240d09424f2b51a15a2aa', 'request_id': 'req-75037e5e-548a-4899-81b0-cafc65f08234'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:39.279338 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-75037e5e-548a-4899-81b0-cafc65f08234 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8cadb52387c240d09424f2b51a15a2aa', 'request_id': 'req-75037e5e-548a-4899-81b0-cafc65f08234'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:39.286258 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-72c87af8-0708-49a2-9af0-89578959313f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07d1e046c9be49d7a889f4461ba35ca5'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:39.290879 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-75037e5e-548a-4899-81b0-cafc65f08234 admin admin] The token cache is being invalidate because project 8cadb52387c240d09424f2b51a15a2aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 17 19:30:39.291277 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-75037e5e-548a-4899-81b0-cafc65f08234 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:39.292006 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-75037e5e-548a-4899-81b0-cafc65f08234 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:39.294043 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-72c87af8-0708-49a2-9af0-89578959313f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07d1e046c9be49d7a889f4461ba35ca5'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:39.294258 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1038/2075] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 17 19:30:38 2022] DELETE /identity/v3/projects/8cadb52387c240d09424f2b51a15a2aa => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:30:39.295015 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75037e5e-548a-4899-81b0-cafc65f08234 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:39.296290 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-75037e5e-548a-4899-81b0-cafc65f08234 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:39.308397 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-72c87af8-0708-49a2-9af0-89578959313f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07d1e046c9be49d7a889f4461ba35ca5'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:39.309128 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-72c87af8-0708-49a2-9af0-89578959313f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07d1e046c9be49d7a889f4461ba35ca5'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:39.309931 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-72c87af8-0708-49a2-9af0-89578959313f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07d1e046c9be49d7a889f4461ba35ca5'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:39.355881 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75037e5e-548a-4899-81b0-cafc65f08234 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:39.359540 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def2e78e-eaf4-4fb6-b547-3a042618d6cb service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57e888884d2b4629bfd12d1034ce5be4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:39.360177 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def2e78e-eaf4-4fb6-b547-3a042618d6cb service glance] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:39.360452 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def2e78e-eaf4-4fb6-b547-3a042618d6cb service glance] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:39.360790 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def2e78e-eaf4-4fb6-b547-3a042618d6cb service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:39.362460 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-def2e78e-eaf4-4fb6-b547-3a042618d6cb service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:39.422261 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def2e78e-eaf4-4fb6-b547-3a042618d6cb service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:39.423743 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def2e78e-eaf4-4fb6-b547-3a042618d6cb service glance] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:39.431978 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1038/2076] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 17 19:30:38 2022] DELETE /identity/v3/users/07d1e046c9be49d7a889f4461ba35ca5 => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:30:39.433522 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e0a72e-bdfc-4023-bfcb-df3f92048f98 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:39.434122 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e0a72e-bdfc-4023-bfcb-df3f92048f98 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:39.434690 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e0a72e-bdfc-4023-bfcb-df3f92048f98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:39.439255 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1039/2077] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:30:39 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:30:39.456312 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-68e0a72e-bdfc-4023-bfcb-df3f92048f98 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:30:39.457505 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-68e0a72e-bdfc-4023-bfcb-df3f92048f98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:39.530784 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1039/2078] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:30:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:30:39.539385 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb29e97c-5184-41ad-a948-c4131b49d305 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:39.539992 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb29e97c-5184-41ad-a948-c4131b49d305 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:39.540696 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb29e97c-5184-41ad-a948-c4131b49d305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:39.566889 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-bb29e97c-5184-41ad-a948-c4131b49d305 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:30:39.569047 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-bb29e97c-5184-41ad-a948-c4131b49d305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:39.587332 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1040/2079] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:30:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:30:39.592180 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e0a72e-bdfc-4023-bfcb-df3f92048f98 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:39.602822 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e0a72e-bdfc-4023-bfcb-df3f92048f98 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:39.608290 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6911dcd-44a7-4535-9b79-39bd39f735b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:39.609812 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6911dcd-44a7-4535-9b79-39bd39f735b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:39.609812 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6911dcd-44a7-4535-9b79-39bd39f735b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:39.609812 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6911dcd-44a7-4535-9b79-39bd39f735b0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:39.610237 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6911dcd-44a7-4535-9b79-39bd39f735b0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:30:39.610539 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6911dcd-44a7-4535-9b79-39bd39f735b0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:39.612239 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6911dcd-44a7-4535-9b79-39bd39f735b0 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:39.624353 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1040/2080] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 17 19:30:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:30:39.629072 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb29e97c-5184-41ad-a948-c4131b49d305 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:39.641409 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb29e97c-5184-41ad-a948-c4131b49d305 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:39.647391 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbb8cb7-61fe-4d26-89cd-a6dda38773ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:39.648338 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbb8cb7-61fe-4d26-89cd-a6dda38773ff admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:39.648726 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbb8cb7-61fe-4d26-89cd-a6dda38773ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:39.648990 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbb8cb7-61fe-4d26-89cd-a6dda38773ff admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:39.649705 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbb8cb7-61fe-4d26-89cd-a6dda38773ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:39.651813 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbb8cb7-61fe-4d26-89cd-a6dda38773ff admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:39.776692 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1041/2081] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 17 19:30:39 2022] POST /identity/v3/projects => generated 361 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:30:39.781417 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6911dcd-44a7-4535-9b79-39bd39f735b0 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:39.802843 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6911dcd-44a7-4535-9b79-39bd39f735b0 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:39.936057 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a03c6e-b6aa-4a64-ad63-9ef22275548e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:39.941437 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a03c6e-b6aa-4a64-ad63-9ef22275548e admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:39.941437 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a03c6e-b6aa-4a64-ad63-9ef22275548e admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:39.941437 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a03c6e-b6aa-4a64-ad63-9ef22275548e admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:39.941437 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a03c6e-b6aa-4a64-ad63-9ef22275548e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:39.941437 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a03c6e-b6aa-4a64-ad63-9ef22275548e admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:39.993470 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbb8cb7-61fe-4d26-89cd-a6dda38773ff admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:39.995450 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-2bbb8cb7-61fe-4d26-89cd-a6dda38773ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:40.013790 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbb8cb7-61fe-4d26-89cd-a6dda38773ff admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:40.022665 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1041/2082] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 17 19:30:39 2022] POST /identity/v3/users => generated 281 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:30:40.028530 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a03c6e-b6aa-4a64-ad63-9ef22275548e admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:40.037171 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867766b0-49fb-45a6-9007-8307eb74092d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:40.037171 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867766b0-49fb-45a6-9007-8307eb74092d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:40.037171 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867766b0-49fb-45a6-9007-8307eb74092d admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:40.037171 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867766b0-49fb-45a6-9007-8307eb74092d admin admin] PATH_INFO: `/v3/projects/a246b8459f3243019886b94e9f4288d7` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:40.037659 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867766b0-49fb-45a6-9007-8307eb74092d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a246b8459f3243019886b94e9f4288d7)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:40.040375 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867766b0-49fb-45a6-9007-8307eb74092d admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:40.040835 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a03c6e-b6aa-4a64-ad63-9ef22275548e admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:40.046548 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a56e20a-fd1e-497a-a67f-ea94ea1ad36f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:40.047354 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a56e20a-fd1e-497a-a67f-ea94ea1ad36f admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:40.047719 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a56e20a-fd1e-497a-a67f-ea94ea1ad36f admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:40.048060 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a56e20a-fd1e-497a-a67f-ea94ea1ad36f admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:40.048833 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a56e20a-fd1e-497a-a67f-ea94ea1ad36f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:40.050503 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a56e20a-fd1e-497a-a67f-ea94ea1ad36f admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:40.063156 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1042/2083] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:30:40 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:30:40.067693 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a56e20a-fd1e-497a-a67f-ea94ea1ad36f admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:40.078312 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a56e20a-fd1e-497a-a67f-ea94ea1ad36f admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:40.083033 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8caf59-d9cf-4673-a853-05e9c2295029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:40.088231 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8caf59-d9cf-4673-a853-05e9c2295029 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:40.088231 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8caf59-d9cf-4673-a853-05e9c2295029 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:40.088231 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8caf59-d9cf-4673-a853-05e9c2295029 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:40.088231 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8caf59-d9cf-4673-a853-05e9c2295029 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:40.088231 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8caf59-d9cf-4673-a853-05e9c2295029 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:40.097903 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1043/2084] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:30:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:30:40.101930 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8caf59-d9cf-4673-a853-05e9c2295029 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:40.112617 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8caf59-d9cf-4673-a853-05e9c2295029 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:40.117291 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0d0951-d2ba-4f06-adc1-443032ae23ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:40.118466 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0d0951-d2ba-4f06-adc1-443032ae23ce admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:40.118746 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0d0951-d2ba-4f06-adc1-443032ae23ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:40.119016 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0d0951-d2ba-4f06-adc1-443032ae23ce admin admin] PATH_INFO: `/v3/projects/b2108a1368934bde865bb0903c96d082/users/9b8a2389961548f1a0ebc90b0bd4feff/roles/114e6a16e63447d1924707ace5aace25` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:40.135081 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0d0951-d2ba-4f06-adc1-443032ae23ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2108a1368934bde865bb0903c96d082, user_id=9b8a2389961548f1a0ebc90b0bd4feff, role_id=114e6a16e63447d1924707ace5aace25)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:40.137811 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0d0951-d2ba-4f06-adc1-443032ae23ce admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:40.268535 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1044/2085] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:30:40 2022] PUT /identity/v3/projects/b2108a1368934bde865bb0903c96d082/users/9b8a2389961548f1a0ebc90b0bd4feff/roles/114e6a16e63447d1924707ace5aace25 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 17 19:30:40.270435 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3d257c-7a38-450e-bdc1-444e4b8e5d50 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:40.270725 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3d257c-7a38-450e-bdc1-444e4b8e5d50 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:40.271001 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3d257c-7a38-450e-bdc1-444e4b8e5d50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:40.271339 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-867766b0-49fb-45a6-9007-8307eb74092d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a246b8459f3243019886b94e9f4288d7', 'request_id': 'req-867766b0-49fb-45a6-9007-8307eb74092d'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:40.281244 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-867766b0-49fb-45a6-9007-8307eb74092d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a246b8459f3243019886b94e9f4288d7', 'request_id': 'req-867766b0-49fb-45a6-9007-8307eb74092d'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:40.301751 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-9d3d257c-7a38-450e-bdc1-444e4b8e5d50 None None] MFA Rules not processed for user `9b8a2389961548f1a0ebc90b0bd4feff`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:30:40.303102 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-9d3d257c-7a38-450e-bdc1-444e4b8e5d50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:40.387066 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-867766b0-49fb-45a6-9007-8307eb74092d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a246b8459f3243019886b94e9f4288d7', 'request_id': 'req-867766b0-49fb-45a6-9007-8307eb74092d'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:40.408135 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1045/2086] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:30:40 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 17 19:30:40.439724 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-867766b0-49fb-45a6-9007-8307eb74092d admin admin] The token cache is being invalidate because project a246b8459f3243019886b94e9f4288d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 17 19:30:40.440024 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-867766b0-49fb-45a6-9007-8307eb74092d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:40.442414 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-867766b0-49fb-45a6-9007-8307eb74092d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:40.449200 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1042/2087] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 17 19:30:39 2022] DELETE /identity/v3/projects/a246b8459f3243019886b94e9f4288d7 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 17 19:30:40.449491 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3d257c-7a38-450e-bdc1-444e4b8e5d50 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:40.451027 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-9d3d257c-7a38-450e-bdc1-444e4b8e5d50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:40.455038 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875fe1e9-6453-441d-8e20-d85aedd5c74a None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:40.459023 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875fe1e9-6453-441d-8e20-d85aedd5c74a None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:40.459023 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875fe1e9-6453-441d-8e20-d85aedd5c74a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:40.484479 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-875fe1e9-6453-441d-8e20-d85aedd5c74a None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:30:40.488135 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-875fe1e9-6453-441d-8e20-d85aedd5c74a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:40.615812 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3d257c-7a38-450e-bdc1-444e4b8e5d50 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:40.635365 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba25db0-3be5-4ad2-93b1-e6da0ec0f0a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd05409b38ae346458abfe41ebda1c59e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:40.635365 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba25db0-3be5-4ad2-93b1-e6da0ec0f0a8 service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:40.635365 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba25db0-3be5-4ad2-93b1-e6da0ec0f0a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:40.635365 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba25db0-3be5-4ad2-93b1-e6da0ec0f0a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:40.636059 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-8ba25db0-3be5-4ad2-93b1-e6da0ec0f0a8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:40.658547 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1043/2088] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:30:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:30:40.666785 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb00bf1-ca78-4aab-a579-074c5d44aa27 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:40.667100 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb00bf1-ca78-4aab-a579-074c5d44aa27 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:40.667351 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb00bf1-ca78-4aab-a579-074c5d44aa27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:40.697075 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-feb00bf1-ca78-4aab-a579-074c5d44aa27 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:30:40.698322 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-feb00bf1-ca78-4aab-a579-074c5d44aa27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:40.709353 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1044/2089] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:30:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:30:40.710865 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb00bf1-ca78-4aab-a579-074c5d44aa27 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:40.712405 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-feb00bf1-ca78-4aab-a579-074c5d44aa27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:40.735723 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb00bf1-ca78-4aab-a579-074c5d44aa27 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:40.742182 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec8984c-88d2-4e5f-9618-692588496a9e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd05409b38ae346458abfe41ebda1c59e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:40.742945 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec8984c-88d2-4e5f-9618-692588496a9e service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:40.743228 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec8984c-88d2-4e5f-9618-692588496a9e service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:40.743603 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec8984c-88d2-4e5f-9618-692588496a9e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:40.748645 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-9ec8984c-88d2-4e5f-9618-692588496a9e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:40.754087 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba25db0-3be5-4ad2-93b1-e6da0ec0f0a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:40.755561 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba25db0-3be5-4ad2-93b1-e6da0ec0f0a8 service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:40.758321 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec8984c-88d2-4e5f-9618-692588496a9e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:40.759802 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec8984c-88d2-4e5f-9618-692588496a9e service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:40.766076 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1046/2090] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:30:40 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 17 19:30:40.766851 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba25db0-3be5-4ad2-93b1-e6da0ec0f0a8 service cinder] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:40.775039 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1045/2091] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:30:40 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:30:40.779676 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba25db0-3be5-4ad2-93b1-e6da0ec0f0a8 service cinder] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:40.783639 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec8984c-88d2-4e5f-9618-692588496a9e service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:40.784951 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-9ec8984c-88d2-4e5f-9618-692588496a9e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:40.785674 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaffe393-1c5f-4a8f-97a8-7f4c0d71e5e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:40.786359 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaffe393-1c5f-4a8f-97a8-7f4c0d71e5e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:40.786641 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaffe393-1c5f-4a8f-97a8-7f4c0d71e5e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:40.786916 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaffe393-1c5f-4a8f-97a8-7f4c0d71e5e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:40.787805 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaffe393-1c5f-4a8f-97a8-7f4c0d71e5e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:30:40.788111 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaffe393-1c5f-4a8f-97a8-7f4c0d71e5e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:40.789711 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaffe393-1c5f-4a8f-97a8-7f4c0d71e5e3 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:40.799304 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec8984c-88d2-4e5f-9618-692588496a9e service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:40.803216 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1047/2092] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 17 19:30:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:30:40.806923 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5af9f1-9467-4273-90f4-e58263b8b8db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:40.807359 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaffe393-1c5f-4a8f-97a8-7f4c0d71e5e3 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:40.808050 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5af9f1-9467-4273-90f4-e58263b8b8db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:40.808402 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5af9f1-9467-4273-90f4-e58263b8b8db admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:40.808744 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5af9f1-9467-4273-90f4-e58263b8b8db admin admin] PATH_INFO: `/v3/users/9b8a2389961548f1a0ebc90b0bd4feff` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:40.810147 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5af9f1-9467-4273-90f4-e58263b8b8db admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b8a2389961548f1a0ebc90b0bd4feff)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:40.812260 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5af9f1-9467-4273-90f4-e58263b8b8db admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:40.818342 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaffe393-1c5f-4a8f-97a8-7f4c0d71e5e3 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:40.823357 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1629cb-406e-4fb0-a2d1-788b37e20e8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:40.824572 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1629cb-406e-4fb0-a2d1-788b37e20e8e admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:40.824858 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1629cb-406e-4fb0-a2d1-788b37e20e8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:40.825128 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1629cb-406e-4fb0-a2d1-788b37e20e8e admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:40.825817 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1629cb-406e-4fb0-a2d1-788b37e20e8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:40.827734 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1629cb-406e-4fb0-a2d1-788b37e20e8e admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:40.907457 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1048/2093] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 17 19:30:40 2022] POST /identity/v3/projects => generated 385 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:30:40.913141 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1629cb-406e-4fb0-a2d1-788b37e20e8e admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:40.988061 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1629cb-406e-4fb0-a2d1-788b37e20e8e admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:40.994472 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dba543-1a8f-48c6-87ab-f16c57c6031b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:40.995236 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dba543-1a8f-48c6-87ab-f16c57c6031b admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:40.995619 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dba543-1a8f-48c6-87ab-f16c57c6031b admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:40.996018 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dba543-1a8f-48c6-87ab-f16c57c6031b admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:40.996830 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dba543-1a8f-48c6-87ab-f16c57c6031b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:40.999054 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dba543-1a8f-48c6-87ab-f16c57c6031b admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:41.011000 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ca5af9f1-9467-4273-90f4-e58263b8b8db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b8a2389961548f1a0ebc90b0bd4feff'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:41.015029 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ca5af9f1-9467-4273-90f4-e58263b8b8db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b8a2389961548f1a0ebc90b0bd4feff'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:41.027090 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ca5af9f1-9467-4273-90f4-e58263b8b8db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b8a2389961548f1a0ebc90b0bd4feff'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:41.160408 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1049/2094] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 17 19:30:40 2022] POST /identity/v3/users => generated 293 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 17 19:30:41.161641 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dba543-1a8f-48c6-87ab-f16c57c6031b admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:41.163228 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-e8dba543-1a8f-48c6-87ab-f16c57c6031b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:41.176840 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dba543-1a8f-48c6-87ab-f16c57c6031b admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:41.182016 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1e1aa3-7b2a-499a-aa57-436f0fb027ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:41.182679 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1e1aa3-7b2a-499a-aa57-436f0fb027ae admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:41.182953 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1e1aa3-7b2a-499a-aa57-436f0fb027ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:41.183218 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1e1aa3-7b2a-499a-aa57-436f0fb027ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:41.184732 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1e1aa3-7b2a-499a-aa57-436f0fb027ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:41.193822 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1e1aa3-7b2a-499a-aa57-436f0fb027ae admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:41.196631 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1050/2095] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:30:41 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 17 19:30:41.200345 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1e1aa3-7b2a-499a-aa57-436f0fb027ae admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:41.210061 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1e1aa3-7b2a-499a-aa57-436f0fb027ae admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:41.215423 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27109f2c-b47b-459a-82b7-85163972fc5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:41.216092 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27109f2c-b47b-459a-82b7-85163972fc5f admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:41.216368 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27109f2c-b47b-459a-82b7-85163972fc5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:41.216635 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27109f2c-b47b-459a-82b7-85163972fc5f admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:41.217336 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27109f2c-b47b-459a-82b7-85163972fc5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:41.219940 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ca5af9f1-9467-4273-90f4-e58263b8b8db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b8a2389961548f1a0ebc90b0bd4feff'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:41.226164 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27109f2c-b47b-459a-82b7-85163972fc5f admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:41.228648 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ca5af9f1-9467-4273-90f4-e58263b8b8db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b8a2389961548f1a0ebc90b0bd4feff'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:41.242538 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1046/2096] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 17 19:30:40 2022] DELETE /identity/v3/users/9b8a2389961548f1a0ebc90b0bd4feff => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 17 19:30:41.247087 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6b9ff7-556d-4b96-ade0-644582f0d72a None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:41.247375 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6b9ff7-556d-4b96-ade0-644582f0d72a None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:41.247669 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6b9ff7-556d-4b96-ade0-644582f0d72a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:41.252553 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1051/2097] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:30:41 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 17 19:30:41.254433 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27109f2c-b47b-459a-82b7-85163972fc5f admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:41.265823 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27109f2c-b47b-459a-82b7-85163972fc5f admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:41.267534 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-0b6b9ff7-556d-4b96-ade0-644582f0d72a None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:30:41.269027 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-0b6b9ff7-556d-4b96-ade0-644582f0d72a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:41.360619 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1047/2098] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:30:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 17 19:30:41.372221 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6b9ff7-556d-4b96-ade0-644582f0d72a None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:41.373550 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58d36f3-cc59-4770-b8ec-c81ff9a80d6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:41.373681 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-0b6b9ff7-556d-4b96-ade0-644582f0d72a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:41.377594 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58d36f3-cc59-4770-b8ec-c81ff9a80d6f admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:41.378159 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58d36f3-cc59-4770-b8ec-c81ff9a80d6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:41.378618 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58d36f3-cc59-4770-b8ec-c81ff9a80d6f admin admin] PATH_INFO: `/v3/projects/e9b5d2193ca54fdeaf28ec699a99fc26/users/aaa2fc269277455a952424d51817032f/roles/114e6a16e63447d1924707ace5aace25` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:41.419176 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58d36f3-cc59-4770-b8ec-c81ff9a80d6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9b5d2193ca54fdeaf28ec699a99fc26, user_id=aaa2fc269277455a952424d51817032f, role_id=114e6a16e63447d1924707ace5aace25)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:41.426087 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58d36f3-cc59-4770-b8ec-c81ff9a80d6f admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:41.489612 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6b9ff7-556d-4b96-ade0-644582f0d72a None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:41.494217 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8cee30-c81b-4fcf-85a3-78ac34bfee0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:41.495063 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8cee30-c81b-4fcf-85a3-78ac34bfee0d service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:41.495342 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8cee30-c81b-4fcf-85a3-78ac34bfee0d service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:41.495669 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8cee30-c81b-4fcf-85a3-78ac34bfee0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:41.510690 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8cee30-c81b-4fcf-85a3-78ac34bfee0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:41.512806 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8cee30-c81b-4fcf-85a3-78ac34bfee0d service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:41.543386 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1048/2099] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:30:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:30:41.620684 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1052/2100] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:30:41 2022] PUT /identity/v3/projects/e9b5d2193ca54fdeaf28ec699a99fc26/users/aaa2fc269277455a952424d51817032f/roles/114e6a16e63447d1924707ace5aace25 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:30:41.636549 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2947329-6ec7-416a-be37-07892490458c None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:41.636836 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2947329-6ec7-416a-be37-07892490458c None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:41.637670 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2947329-6ec7-416a-be37-07892490458c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:41.674757 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-f2947329-6ec7-416a-be37-07892490458c None None] MFA Rules not processed for user `aaa2fc269277455a952424d51817032f`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:30:41.676748 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-f2947329-6ec7-416a-be37-07892490458c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:41.790002 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1049/2101] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:30:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:30:41.803266 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58d36f3-cc59-4770-b8ec-c81ff9a80d6f admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:41.806475 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-c58d36f3-cc59-4770-b8ec-c81ff9a80d6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:41.904092 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58d36f3-cc59-4770-b8ec-c81ff9a80d6f admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:41.907963 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a0f478-9181-43b5-9034-98d212e0695e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd05409b38ae346458abfe41ebda1c59e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:41.909378 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a0f478-9181-43b5-9034-98d212e0695e service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:41.909378 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a0f478-9181-43b5-9034-98d212e0695e service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:41.909378 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a0f478-9181-43b5-9034-98d212e0695e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:41.927349 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a0f478-9181-43b5-9034-98d212e0695e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:41.929219 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a0f478-9181-43b5-9034-98d212e0695e service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:41.947529 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1053/2102] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:30:41 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:30:42.120593 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2947329-6ec7-416a-be37-07892490458c None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:42.122763 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-f2947329-6ec7-416a-be37-07892490458c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:42.198378 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2947329-6ec7-416a-be37-07892490458c None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:42.203221 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c532a88-40d3-41dc-9631-06abc19ff10c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:42.204127 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c532a88-40d3-41dc-9631-06abc19ff10c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:42.205029 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c532a88-40d3-41dc-9631-06abc19ff10c admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:42.205370 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c532a88-40d3-41dc-9631-06abc19ff10c admin admin] PATH_INFO: `/v3/projects/b2108a1368934bde865bb0903c96d082` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:42.206620 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c532a88-40d3-41dc-9631-06abc19ff10c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2108a1368934bde865bb0903c96d082)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:42.208639 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c532a88-40d3-41dc-9631-06abc19ff10c admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:42.311153 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-5c532a88-40d3-41dc-9631-06abc19ff10c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2108a1368934bde865bb0903c96d082', 'request_id': 'req-5c532a88-40d3-41dc-9631-06abc19ff10c'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:42.316463 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-5c532a88-40d3-41dc-9631-06abc19ff10c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2108a1368934bde865bb0903c96d082', 'request_id': 'req-5c532a88-40d3-41dc-9631-06abc19ff10c'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:42.398722 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-5c532a88-40d3-41dc-9631-06abc19ff10c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2108a1368934bde865bb0903c96d082', 'request_id': 'req-5c532a88-40d3-41dc-9631-06abc19ff10c'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:42.409888 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-5c532a88-40d3-41dc-9631-06abc19ff10c admin admin] The token cache is being invalidate because project b2108a1368934bde865bb0903c96d082 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 17 19:30:42.410169 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-5c532a88-40d3-41dc-9631-06abc19ff10c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:42.410778 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-5c532a88-40d3-41dc-9631-06abc19ff10c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:42.412488 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1050/2103] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 17 19:30:42 2022] DELETE /identity/v3/projects/b2108a1368934bde865bb0903c96d082 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:30:42.417269 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeace86-0648-483e-b7cd-edbd8c537a89 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:42.417552 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeace86-0648-483e-b7cd-edbd8c537a89 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:42.418314 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeace86-0648-483e-b7cd-edbd8c537a89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:42.436169 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-ebeace86-0648-483e-b7cd-edbd8c537a89 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:30:42.437350 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-ebeace86-0648-483e-b7cd-edbd8c537a89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:42.494235 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1054/2104] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:30:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 17 19:30:42.504074 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4520b0e-b296-4796-b5fd-ad8533a5faab None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:42.504356 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4520b0e-b296-4796-b5fd-ad8533a5faab None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:42.504672 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4520b0e-b296-4796-b5fd-ad8533a5faab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:42.522805 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-f4520b0e-b296-4796-b5fd-ad8533a5faab None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:30:42.524437 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-f4520b0e-b296-4796-b5fd-ad8533a5faab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:42.531925 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1051/2105] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:30:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 17 19:30:42.536250 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeace86-0648-483e-b7cd-edbd8c537a89 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:42.546137 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeace86-0648-483e-b7cd-edbd8c537a89 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:42.553574 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3e4961-03d9-4522-bd37-e9c975b99c3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:42.554235 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3e4961-03d9-4522-bd37-e9c975b99c3a admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:42.554512 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3e4961-03d9-4522-bd37-e9c975b99c3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:42.554782 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3e4961-03d9-4522-bd37-e9c975b99c3a admin admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:42.555536 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3e4961-03d9-4522-bd37-e9c975b99c3a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:30:42.555844 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3e4961-03d9-4522-bd37-e9c975b99c3a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:42.557553 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3e4961-03d9-4522-bd37-e9c975b99c3a admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:42.573944 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1055/2106] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 17 19:30:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:30:42.575050 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4520b0e-b296-4796-b5fd-ad8533a5faab None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:42.586550 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4520b0e-b296-4796-b5fd-ad8533a5faab None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:42.591944 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2823964a-7609-49ee-9725-eace350ad05a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:42.592595 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2823964a-7609-49ee-9725-eace350ad05a admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:42.592871 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2823964a-7609-49ee-9725-eace350ad05a admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:42.593141 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2823964a-7609-49ee-9725-eace350ad05a admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:42.593864 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2823964a-7609-49ee-9725-eace350ad05a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:42.596895 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2823964a-7609-49ee-9725-eace350ad05a admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:42.703213 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1052/2107] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 17 19:30:42 2022] POST /identity/v3/projects => generated 373 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:30:42.707463 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3e4961-03d9-4522-bd37-e9c975b99c3a admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:42.717786 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3e4961-03d9-4522-bd37-e9c975b99c3a admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:42.772804 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af11deed-e07f-4ef1-bb1a-6d9eb7f335c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:42.773473 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af11deed-e07f-4ef1-bb1a-6d9eb7f335c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:42.773753 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af11deed-e07f-4ef1-bb1a-6d9eb7f335c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:42.774016 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af11deed-e07f-4ef1-bb1a-6d9eb7f335c5 admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:42.774886 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af11deed-e07f-4ef1-bb1a-6d9eb7f335c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:42.776791 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af11deed-e07f-4ef1-bb1a-6d9eb7f335c5 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:42.838680 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1056/2108] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 17 19:30:42 2022] POST /identity/v3/users => generated 287 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:30:42.843459 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2823964a-7609-49ee-9725-eace350ad05a admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:42.853958 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2823964a-7609-49ee-9725-eace350ad05a admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:42.859460 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331cf240-f3be-45b1-b297-d7a1dae781d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:42.860184 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331cf240-f3be-45b1-b297-d7a1dae781d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:42.860495 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331cf240-f3be-45b1-b297-d7a1dae781d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:42.860765 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331cf240-f3be-45b1-b297-d7a1dae781d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:42.861514 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331cf240-f3be-45b1-b297-d7a1dae781d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:42.863122 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331cf240-f3be-45b1-b297-d7a1dae781d7 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:42.876739 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1053/2109] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:30:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:30:42.884707 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af11deed-e07f-4ef1-bb1a-6d9eb7f335c5 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:42.887996 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af11deed-e07f-4ef1-bb1a-6d9eb7f335c5 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:42.892927 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9724da11-f761-44fa-a4b0-895a05eb5f07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:42.893563 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9724da11-f761-44fa-a4b0-895a05eb5f07 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:42.893837 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9724da11-f761-44fa-a4b0-895a05eb5f07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:42.894106 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9724da11-f761-44fa-a4b0-895a05eb5f07 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:42.894892 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9724da11-f761-44fa-a4b0-895a05eb5f07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:42.896660 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9724da11-f761-44fa-a4b0-895a05eb5f07 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:42.908705 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1057/2110] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:30:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:30:42.913926 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331cf240-f3be-45b1-b297-d7a1dae781d7 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:42.924953 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331cf240-f3be-45b1-b297-d7a1dae781d7 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:42.929695 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49530d9-4e5e-4823-ba58-706fc9326d09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:42.930561 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49530d9-4e5e-4823-ba58-706fc9326d09 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:42.930835 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49530d9-4e5e-4823-ba58-706fc9326d09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:42.931101 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49530d9-4e5e-4823-ba58-706fc9326d09 admin admin] PATH_INFO: `/v3/projects/c0114ecf015847908c81a974202a6595/users/142bf5db756441149656045c802e70e1/roles/114e6a16e63447d1924707ace5aace25` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:42.947707 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49530d9-4e5e-4823-ba58-706fc9326d09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0114ecf015847908c81a974202a6595, user_id=142bf5db756441149656045c802e70e1, role_id=114e6a16e63447d1924707ace5aace25)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:42.949312 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49530d9-4e5e-4823-ba58-706fc9326d09 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:43.070958 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1054/2111] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:30:42 2022] PUT /identity/v3/projects/c0114ecf015847908c81a974202a6595/users/142bf5db756441149656045c802e70e1/roles/114e6a16e63447d1924707ace5aace25 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:30:43.081990 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec68ac5-8bb9-46dc-9dd4-cef6c2571a27 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:43.082276 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec68ac5-8bb9-46dc-9dd4-cef6c2571a27 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:43.082575 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec68ac5-8bb9-46dc-9dd4-cef6c2571a27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:43.109829 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-5ec68ac5-8bb9-46dc-9dd4-cef6c2571a27 None None] MFA Rules not processed for user `142bf5db756441149656045c802e70e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:30:43.110990 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-5ec68ac5-8bb9-46dc-9dd4-cef6c2571a27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:43.176960 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1058/2112] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:30:43 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:30:43.194390 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49530d9-4e5e-4823-ba58-706fc9326d09 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:43.196869 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c49530d9-4e5e-4823-ba58-706fc9326d09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:43.252825 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49530d9-4e5e-4823-ba58-706fc9326d09 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:43.257158 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c13c6a-17cc-4e9d-b12e-231419cb5094 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd05409b38ae346458abfe41ebda1c59e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:43.257842 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c13c6a-17cc-4e9d-b12e-231419cb5094 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:43.258132 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c13c6a-17cc-4e9d-b12e-231419cb5094 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:43.258383 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c13c6a-17cc-4e9d-b12e-231419cb5094 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:43.267057 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c13c6a-17cc-4e9d-b12e-231419cb5094 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:43.268538 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c13c6a-17cc-4e9d-b12e-231419cb5094 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:43.277534 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1055/2113] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:30:43 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:30:47.216354 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec68ac5-8bb9-46dc-9dd4-cef6c2571a27 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:47.217804 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-5ec68ac5-8bb9-46dc-9dd4-cef6c2571a27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:47.279890 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec68ac5-8bb9-46dc-9dd4-cef6c2571a27 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:47.284745 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32781c3d-9112-475b-bd80-ae85d9b46894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:47.285293 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32781c3d-9112-475b-bd80-ae85d9b46894 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:47.285573 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32781c3d-9112-475b-bd80-ae85d9b46894 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:47.285844 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32781c3d-9112-475b-bd80-ae85d9b46894 admin admin] PATH_INFO: `/v3/users/fc9129cb9f874b32b99c78349dd58477` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:47.287048 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32781c3d-9112-475b-bd80-ae85d9b46894 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc9129cb9f874b32b99c78349dd58477)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:47.288996 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32781c3d-9112-475b-bd80-ae85d9b46894 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:47.474700 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-32781c3d-9112-475b-bd80-ae85d9b46894 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc9129cb9f874b32b99c78349dd58477'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:47.481318 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-32781c3d-9112-475b-bd80-ae85d9b46894 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc9129cb9f874b32b99c78349dd58477'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:47.492378 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-32781c3d-9112-475b-bd80-ae85d9b46894 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc9129cb9f874b32b99c78349dd58477'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:47.494473 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-32781c3d-9112-475b-bd80-ae85d9b46894 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc9129cb9f874b32b99c78349dd58477'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:47.494473 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-32781c3d-9112-475b-bd80-ae85d9b46894 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc9129cb9f874b32b99c78349dd58477'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:47.619098 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1059/2114] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 17 19:30:47 2022] DELETE /identity/v3/users/fc9129cb9f874b32b99c78349dd58477 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:30:47.875400 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c13c6a-17cc-4e9d-b12e-231419cb5094 service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:47.889507 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c13c6a-17cc-4e9d-b12e-231419cb5094 service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:47.946375 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cb17cb-0329-4288-813a-ff1849dfca47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:47.947235 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cb17cb-0329-4288-813a-ff1849dfca47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:47.947536 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cb17cb-0329-4288-813a-ff1849dfca47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:47.947813 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cb17cb-0329-4288-813a-ff1849dfca47 admin admin] PATH_INFO: `/v3/projects/9201351f65334032be89ac50d14926ba` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:47.949589 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cb17cb-0329-4288-813a-ff1849dfca47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9201351f65334032be89ac50d14926ba)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:47.951528 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cb17cb-0329-4288-813a-ff1849dfca47 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:48.064062 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-67cb17cb-0329-4288-813a-ff1849dfca47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9201351f65334032be89ac50d14926ba', 'request_id': 'req-67cb17cb-0329-4288-813a-ff1849dfca47'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:48.069204 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-67cb17cb-0329-4288-813a-ff1849dfca47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9201351f65334032be89ac50d14926ba', 'request_id': 'req-67cb17cb-0329-4288-813a-ff1849dfca47'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:48.167994 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-67cb17cb-0329-4288-813a-ff1849dfca47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9201351f65334032be89ac50d14926ba', 'request_id': 'req-67cb17cb-0329-4288-813a-ff1849dfca47'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:48.178896 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-67cb17cb-0329-4288-813a-ff1849dfca47 admin admin] The token cache is being invalidate because project 9201351f65334032be89ac50d14926ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 17 19:30:48.179176 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-67cb17cb-0329-4288-813a-ff1849dfca47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:48.179803 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-67cb17cb-0329-4288-813a-ff1849dfca47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:48.181313 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1056/2115] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 17 19:30:47 2022] DELETE /identity/v3/projects/9201351f65334032be89ac50d14926ba => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:30:48.187662 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a77e98-d413-419b-936a-4d287d4033b0 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:48.187963 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a77e98-d413-419b-936a-4d287d4033b0 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:48.188223 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a77e98-d413-419b-936a-4d287d4033b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:48.206343 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-78a77e98-d413-419b-936a-4d287d4033b0 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:30:48.207587 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-78a77e98-d413-419b-936a-4d287d4033b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:48.263952 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1060/2116] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:30:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:30:48.271705 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98edf4d0-e974-45f1-931a-8269170a7a3f None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:48.271988 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98edf4d0-e974-45f1-931a-8269170a7a3f None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:48.272264 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98edf4d0-e974-45f1-931a-8269170a7a3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:48.290183 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-98edf4d0-e974-45f1-931a-8269170a7a3f None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:30:48.291414 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-98edf4d0-e974-45f1-931a-8269170a7a3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:48.299243 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1057/2117] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:30:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:30:48.303190 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a77e98-d413-419b-936a-4d287d4033b0 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:48.313226 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a77e98-d413-419b-936a-4d287d4033b0 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:48.317793 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab625b0e-8ad9-4649-8d67-9dfa2a3c9138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:48.318440 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab625b0e-8ad9-4649-8d67-9dfa2a3c9138 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:48.318718 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab625b0e-8ad9-4649-8d67-9dfa2a3c9138 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:48.318987 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab625b0e-8ad9-4649-8d67-9dfa2a3c9138 admin admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:48.319738 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab625b0e-8ad9-4649-8d67-9dfa2a3c9138 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:30:48.320177 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab625b0e-8ad9-4649-8d67-9dfa2a3c9138 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:48.321767 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab625b0e-8ad9-4649-8d67-9dfa2a3c9138 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:48.336980 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1061/2118] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 17 19:30:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:30:48.338051 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98edf4d0-e974-45f1-931a-8269170a7a3f None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:48.347414 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98edf4d0-e974-45f1-931a-8269170a7a3f None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:48.352173 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a85346e-5a7b-417d-a4ee-4f0e6f5b0557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:48.352812 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a85346e-5a7b-417d-a4ee-4f0e6f5b0557 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:48.353096 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a85346e-5a7b-417d-a4ee-4f0e6f5b0557 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:48.353359 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a85346e-5a7b-417d-a4ee-4f0e6f5b0557 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:48.354036 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a85346e-5a7b-417d-a4ee-4f0e6f5b0557 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:48.356474 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a85346e-5a7b-417d-a4ee-4f0e6f5b0557 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:48.473034 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1058/2119] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 17 19:30:48 2022] POST /identity/v3/projects => generated 385 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:30:48.477149 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab625b0e-8ad9-4649-8d67-9dfa2a3c9138 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:48.487374 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab625b0e-8ad9-4649-8d67-9dfa2a3c9138 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:48.541030 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e4c807-9742-4a6e-90bd-fe563e788549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:48.541686 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e4c807-9742-4a6e-90bd-fe563e788549 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:48.541966 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e4c807-9742-4a6e-90bd-fe563e788549 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:48.542778 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e4c807-9742-4a6e-90bd-fe563e788549 admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:48.543477 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e4c807-9742-4a6e-90bd-fe563e788549 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:48.545398 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e4c807-9742-4a6e-90bd-fe563e788549 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:48.630365 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1062/2120] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 17 19:30:48 2022] POST /identity/v3/users => generated 293 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 17 19:30:48.634371 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a85346e-5a7b-417d-a4ee-4f0e6f5b0557 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:48.644779 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a85346e-5a7b-417d-a4ee-4f0e6f5b0557 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:48.649359 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52d7e6b-f530-4c4b-a243-196316a46e05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:48.649997 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52d7e6b-f530-4c4b-a243-196316a46e05 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:48.650270 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52d7e6b-f530-4c4b-a243-196316a46e05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:48.650538 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52d7e6b-f530-4c4b-a243-196316a46e05 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:48.651283 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52d7e6b-f530-4c4b-a243-196316a46e05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:48.652908 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52d7e6b-f530-4c4b-a243-196316a46e05 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:48.662932 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1059/2121] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:30:48 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:30:48.666714 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e4c807-9742-4a6e-90bd-fe563e788549 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:48.677083 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e4c807-9742-4a6e-90bd-fe563e788549 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:48.681567 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da45996-237c-4545-9ed9-036468077ac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:48.682203 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da45996-237c-4545-9ed9-036468077ac4 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:48.682479 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da45996-237c-4545-9ed9-036468077ac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:48.682750 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da45996-237c-4545-9ed9-036468077ac4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:48.684137 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da45996-237c-4545-9ed9-036468077ac4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:48.685709 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da45996-237c-4545-9ed9-036468077ac4 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:48.695601 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1063/2122] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:30:48 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:30:48.699648 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52d7e6b-f530-4c4b-a243-196316a46e05 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:48.709229 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52d7e6b-f530-4c4b-a243-196316a46e05 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:48.713797 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ecdae5-9500-405d-b567-a2b5dd5032ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:48.715217 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ecdae5-9500-405d-b567-a2b5dd5032ae admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:48.715515 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ecdae5-9500-405d-b567-a2b5dd5032ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:48.715788 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ecdae5-9500-405d-b567-a2b5dd5032ae admin admin] PATH_INFO: `/v3/projects/8553cad4ce8a4fbd8d87d11f21f53872/users/b209201893dc4b27a7f0d9423a30899d/roles/114e6a16e63447d1924707ace5aace25` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:48.730458 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ecdae5-9500-405d-b567-a2b5dd5032ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=8553cad4ce8a4fbd8d87d11f21f53872, user_id=b209201893dc4b27a7f0d9423a30899d, role_id=114e6a16e63447d1924707ace5aace25)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:48.733152 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ecdae5-9500-405d-b567-a2b5dd5032ae admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:48.815847 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1060/2123] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:30:48 2022] PUT /identity/v3/projects/8553cad4ce8a4fbd8d87d11f21f53872/users/b209201893dc4b27a7f0d9423a30899d/roles/114e6a16e63447d1924707ace5aace25 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:30:48.826206 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604f9271-5684-41c0-a728-eaca9d6fcdac None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:48.826500 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604f9271-5684-41c0-a728-eaca9d6fcdac None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:48.826782 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604f9271-5684-41c0-a728-eaca9d6fcdac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:48.854608 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-604f9271-5684-41c0-a728-eaca9d6fcdac None None] MFA Rules not processed for user `b209201893dc4b27a7f0d9423a30899d`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:30:48.856439 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-604f9271-5684-41c0-a728-eaca9d6fcdac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:48.923570 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1064/2124] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:30:48 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:30:48.933359 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ecdae5-9500-405d-b567-a2b5dd5032ae admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:48.935304 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-f8ecdae5-9500-405d-b567-a2b5dd5032ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:49.013850 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ecdae5-9500-405d-b567-a2b5dd5032ae admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:49.019193 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd63c7b4-a987-4519-a98b-cdae58a3733f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c1f878b0dc4d63a4f5dcb12fac1dc1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:49.019852 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd63c7b4-a987-4519-a98b-cdae58a3733f service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:49.020128 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd63c7b4-a987-4519-a98b-cdae58a3733f service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:49.020396 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd63c7b4-a987-4519-a98b-cdae58a3733f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:49.028283 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd63c7b4-a987-4519-a98b-cdae58a3733f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:49.029852 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd63c7b4-a987-4519-a98b-cdae58a3733f service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:49.038486 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1061/2125] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:30:48 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:30:50.938935 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604f9271-5684-41c0-a728-eaca9d6fcdac None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:50.940455 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-604f9271-5684-41c0-a728-eaca9d6fcdac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:51.013418 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604f9271-5684-41c0-a728-eaca9d6fcdac None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:51.017970 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24f3783-1308-4294-be12-ad7c59c19d38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:51.018826 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24f3783-1308-4294-be12-ad7c59c19d38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:51.019100 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24f3783-1308-4294-be12-ad7c59c19d38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:51.019363 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24f3783-1308-4294-be12-ad7c59c19d38 admin admin] PATH_INFO: `/v3/users/142bf5db756441149656045c802e70e1` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:51.023794 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24f3783-1308-4294-be12-ad7c59c19d38 admin admin] RBAC: Authorizing `identity:delete_user(user_id=142bf5db756441149656045c802e70e1)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:51.023794 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24f3783-1308-4294-be12-ad7c59c19d38 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:51.255973 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-d24f3783-1308-4294-be12-ad7c59c19d38 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '142bf5db756441149656045c802e70e1'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:51.263628 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-d24f3783-1308-4294-be12-ad7c59c19d38 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '142bf5db756441149656045c802e70e1'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:51.275179 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-d24f3783-1308-4294-be12-ad7c59c19d38 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '142bf5db756441149656045c802e70e1'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:51.275928 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-d24f3783-1308-4294-be12-ad7c59c19d38 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '142bf5db756441149656045c802e70e1'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:51.276567 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-d24f3783-1308-4294-be12-ad7c59c19d38 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '142bf5db756441149656045c802e70e1'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:51.398510 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1065/2126] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 17 19:30:50 2022] DELETE /identity/v3/users/142bf5db756441149656045c802e70e1 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:30:51.403258 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17c0c3d-fe9d-4c10-bf1f-aa38b9fe3770 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:51.403760 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17c0c3d-fe9d-4c10-bf1f-aa38b9fe3770 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:51.403847 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17c0c3d-fe9d-4c10-bf1f-aa38b9fe3770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:51.421235 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-a17c0c3d-fe9d-4c10-bf1f-aa38b9fe3770 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:30:51.422528 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a17c0c3d-fe9d-4c10-bf1f-aa38b9fe3770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:51.485745 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1062/2127] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:30:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:30:51.502161 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24f3783-1308-4294-be12-ad7c59c19d38 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:51.502161 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-d24f3783-1308-4294-be12-ad7c59c19d38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:51.560449 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24f3783-1308-4294-be12-ad7c59c19d38 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:51.564548 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4664be-607b-4548-ac0e-c45b6d61ef99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:51.565267 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4664be-607b-4548-ac0e-c45b6d61ef99 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:51.565585 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4664be-607b-4548-ac0e-c45b6d61ef99 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:51.565855 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4664be-607b-4548-ac0e-c45b6d61ef99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:51.576553 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4664be-607b-4548-ac0e-c45b6d61ef99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:51.578061 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4664be-607b-4548-ac0e-c45b6d61ef99 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:51.587701 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1066/2128] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:30:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:30:52.075402 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17c0c3d-fe9d-4c10-bf1f-aa38b9fe3770 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:52.076857 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a17c0c3d-fe9d-4c10-bf1f-aa38b9fe3770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:52.094027 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17c0c3d-fe9d-4c10-bf1f-aa38b9fe3770 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:52.099272 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795912e0-c978-486a-a376-4ede56495de4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:52.100198 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795912e0-c978-486a-a376-4ede56495de4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:52.100477 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795912e0-c978-486a-a376-4ede56495de4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:52.100750 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795912e0-c978-486a-a376-4ede56495de4 admin admin] PATH_INFO: `/v3/projects/c0114ecf015847908c81a974202a6595` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:52.101992 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795912e0-c978-486a-a376-4ede56495de4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0114ecf015847908c81a974202a6595)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:52.104058 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795912e0-c978-486a-a376-4ede56495de4 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:52.296282 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-795912e0-c978-486a-a376-4ede56495de4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0114ecf015847908c81a974202a6595', 'request_id': 'req-795912e0-c978-486a-a376-4ede56495de4'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:52.301569 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-795912e0-c978-486a-a376-4ede56495de4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0114ecf015847908c81a974202a6595', 'request_id': 'req-795912e0-c978-486a-a376-4ede56495de4'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:52.403505 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-795912e0-c978-486a-a376-4ede56495de4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0114ecf015847908c81a974202a6595', 'request_id': 'req-795912e0-c978-486a-a376-4ede56495de4'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:52.415838 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-795912e0-c978-486a-a376-4ede56495de4 admin admin] The token cache is being invalidate because project c0114ecf015847908c81a974202a6595 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 17 19:30:52.419843 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-795912e0-c978-486a-a376-4ede56495de4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:52.419843 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-795912e0-c978-486a-a376-4ede56495de4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:52.419843 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1063/2129] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 17 19:30:52 2022] DELETE /identity/v3/projects/c0114ecf015847908c81a974202a6595 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:30:52.424614 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0626c17e-282f-4a9c-9dd6-efe7e057ea9b None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:52.425088 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0626c17e-282f-4a9c-9dd6-efe7e057ea9b None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:52.425302 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0626c17e-282f-4a9c-9dd6-efe7e057ea9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:52.443784 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-0626c17e-282f-4a9c-9dd6-efe7e057ea9b None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:30:52.445248 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-0626c17e-282f-4a9c-9dd6-efe7e057ea9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:52.504626 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1067/2130] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:30:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:30:52.512213 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c17b15-1036-426f-ac9b-215eaf2fbd8f None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:52.512498 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c17b15-1036-426f-ac9b-215eaf2fbd8f None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:52.512769 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c17b15-1036-426f-ac9b-215eaf2fbd8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:52.531572 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-d1c17b15-1036-426f-ac9b-215eaf2fbd8f None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:30:52.532795 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d1c17b15-1036-426f-ac9b-215eaf2fbd8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:52.544857 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1064/2131] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:30:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:30:52.547753 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0626c17e-282f-4a9c-9dd6-efe7e057ea9b None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:52.557221 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0626c17e-282f-4a9c-9dd6-efe7e057ea9b None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:52.561952 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0488b26d-bffd-4cc7-9bfc-b93273d10ba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:52.562596 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0488b26d-bffd-4cc7-9bfc-b93273d10ba7 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:52.562877 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0488b26d-bffd-4cc7-9bfc-b93273d10ba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:52.563147 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0488b26d-bffd-4cc7-9bfc-b93273d10ba7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:52.563901 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0488b26d-bffd-4cc7-9bfc-b93273d10ba7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:30:52.564204 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0488b26d-bffd-4cc7-9bfc-b93273d10ba7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:52.565783 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0488b26d-bffd-4cc7-9bfc-b93273d10ba7 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:52.576932 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1068/2132] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 17 19:30:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:30:52.581289 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c17b15-1036-426f-ac9b-215eaf2fbd8f None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:52.591291 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c17b15-1036-426f-ac9b-215eaf2fbd8f None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:52.595893 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39760a33-9501-4c60-81f6-1b7f68cae1ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:52.596833 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39760a33-9501-4c60-81f6-1b7f68cae1ef admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:52.597109 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39760a33-9501-4c60-81f6-1b7f68cae1ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:52.597384 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39760a33-9501-4c60-81f6-1b7f68cae1ef admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:52.598052 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39760a33-9501-4c60-81f6-1b7f68cae1ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:52.599971 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39760a33-9501-4c60-81f6-1b7f68cae1ef admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:52.718670 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1065/2133] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 17 19:30:52 2022] POST /identity/v3/projects => generated 371 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:30:52.723993 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0488b26d-bffd-4cc7-9bfc-b93273d10ba7 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:52.734016 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0488b26d-bffd-4cc7-9bfc-b93273d10ba7 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:52.791932 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf8385f-2f60-405b-b204-9ab70b0c4574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:52.792599 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf8385f-2f60-405b-b204-9ab70b0c4574 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:52.792938 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf8385f-2f60-405b-b204-9ab70b0c4574 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:52.793452 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf8385f-2f60-405b-b204-9ab70b0c4574 admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:52.794140 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf8385f-2f60-405b-b204-9ab70b0c4574 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:52.796127 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf8385f-2f60-405b-b204-9ab70b0c4574 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:52.902300 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1069/2134] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 17 19:30:52 2022] POST /identity/v3/users => generated 286 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:30:52.906254 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39760a33-9501-4c60-81f6-1b7f68cae1ef admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:52.916799 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39760a33-9501-4c60-81f6-1b7f68cae1ef admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:52.922071 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c3939a-a6cc-4ded-a8a2-90df1000a233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:52.922750 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c3939a-a6cc-4ded-a8a2-90df1000a233 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:52.923029 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c3939a-a6cc-4ded-a8a2-90df1000a233 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:52.923302 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c3939a-a6cc-4ded-a8a2-90df1000a233 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:52.924057 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c3939a-a6cc-4ded-a8a2-90df1000a233 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:52.925659 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c3939a-a6cc-4ded-a8a2-90df1000a233 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:52.937216 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1066/2135] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:30:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:30:52.940991 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf8385f-2f60-405b-b204-9ab70b0c4574 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:52.952160 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf8385f-2f60-405b-b204-9ab70b0c4574 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:52.956991 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88145f3-8b55-4d95-9453-62eef2e69b64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:52.957715 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88145f3-8b55-4d95-9453-62eef2e69b64 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:52.958077 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88145f3-8b55-4d95-9453-62eef2e69b64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:52.958410 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88145f3-8b55-4d95-9453-62eef2e69b64 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:52.959230 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88145f3-8b55-4d95-9453-62eef2e69b64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:52.960915 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88145f3-8b55-4d95-9453-62eef2e69b64 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:52.976494 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1070/2136] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:30:52 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:30:52.977655 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c3939a-a6cc-4ded-a8a2-90df1000a233 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:52.988080 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c3939a-a6cc-4ded-a8a2-90df1000a233 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:52.992919 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a722883f-a3be-4070-aa73-0ca4c25aa2e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:52.993780 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a722883f-a3be-4070-aa73-0ca4c25aa2e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:52.994053 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a722883f-a3be-4070-aa73-0ca4c25aa2e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:52.994326 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a722883f-a3be-4070-aa73-0ca4c25aa2e2 admin admin] PATH_INFO: `/v3/projects/cd0a7a46d0d34dd480e771b2d711242a/users/7eacc38176944eedbd178e43b1e5d035/roles/114e6a16e63447d1924707ace5aace25` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:53.009261 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a722883f-a3be-4070-aa73-0ca4c25aa2e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd0a7a46d0d34dd480e771b2d711242a, user_id=7eacc38176944eedbd178e43b1e5d035, role_id=114e6a16e63447d1924707ace5aace25)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:53.012002 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a722883f-a3be-4070-aa73-0ca4c25aa2e2 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:53.089574 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1067/2137] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:30:52 2022] PUT /identity/v3/projects/cd0a7a46d0d34dd480e771b2d711242a/users/7eacc38176944eedbd178e43b1e5d035/roles/114e6a16e63447d1924707ace5aace25 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:30:53.101084 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df6d176-bdfd-4efd-a2ca-e04faa7f6c9f None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:53.101407 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df6d176-bdfd-4efd-a2ca-e04faa7f6c9f None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:53.101671 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df6d176-bdfd-4efd-a2ca-e04faa7f6c9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:53.127910 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-0df6d176-bdfd-4efd-a2ca-e04faa7f6c9f None None] MFA Rules not processed for user `7eacc38176944eedbd178e43b1e5d035`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:30:53.129103 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-0df6d176-bdfd-4efd-a2ca-e04faa7f6c9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:53.189540 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1071/2138] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:30:53 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:30:53.200947 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a722883f-a3be-4070-aa73-0ca4c25aa2e2 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:53.202619 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a722883f-a3be-4070-aa73-0ca4c25aa2e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:53.256849 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a722883f-a3be-4070-aa73-0ca4c25aa2e2 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:53.260468 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb53e2c-2da7-42b4-8147-0b1dd0ce2b51 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57e888884d2b4629bfd12d1034ce5be4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:53.261127 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb53e2c-2da7-42b4-8147-0b1dd0ce2b51 service glance] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:53.261412 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb53e2c-2da7-42b4-8147-0b1dd0ce2b51 service glance] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:53.261685 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb53e2c-2da7-42b4-8147-0b1dd0ce2b51 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:53.269872 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb53e2c-2da7-42b4-8147-0b1dd0ce2b51 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:53.271296 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb53e2c-2da7-42b4-8147-0b1dd0ce2b51 service glance] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:53.280520 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1068/2139] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:30:53 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 17 19:30:57.049856 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df6d176-bdfd-4efd-a2ca-e04faa7f6c9f None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:57.051294 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-0df6d176-bdfd-4efd-a2ca-e04faa7f6c9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:57.113911 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df6d176-bdfd-4efd-a2ca-e04faa7f6c9f None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:57.118423 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f679ec-8f1e-4f49-b61d-96ee8d674592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:57.119280 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f679ec-8f1e-4f49-b61d-96ee8d674592 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:57.119588 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f679ec-8f1e-4f49-b61d-96ee8d674592 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:57.119859 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f679ec-8f1e-4f49-b61d-96ee8d674592 admin admin] PATH_INFO: `/v3/users/aaa2fc269277455a952424d51817032f` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:57.121100 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f679ec-8f1e-4f49-b61d-96ee8d674592 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aaa2fc269277455a952424d51817032f)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:57.123023 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f679ec-8f1e-4f49-b61d-96ee8d674592 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:57.341969 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-36f679ec-8f1e-4f49-b61d-96ee8d674592 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aaa2fc269277455a952424d51817032f'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:57.348812 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-36f679ec-8f1e-4f49-b61d-96ee8d674592 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aaa2fc269277455a952424d51817032f'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:57.360997 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-36f679ec-8f1e-4f49-b61d-96ee8d674592 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aaa2fc269277455a952424d51817032f'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:57.361709 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-36f679ec-8f1e-4f49-b61d-96ee8d674592 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aaa2fc269277455a952424d51817032f'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:57.362311 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-36f679ec-8f1e-4f49-b61d-96ee8d674592 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aaa2fc269277455a952424d51817032f'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:57.447750 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1072/2140] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 17 19:30:57 2022] DELETE /identity/v3/users/aaa2fc269277455a952424d51817032f => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:30:57.452598 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d156c7c-dfe5-4aaf-a4e8-206a71114338 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:57.453048 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d156c7c-dfe5-4aaf-a4e8-206a71114338 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:57.453147 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d156c7c-dfe5-4aaf-a4e8-206a71114338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:57.470775 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-2d156c7c-dfe5-4aaf-a4e8-206a71114338 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:30:57.472270 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-2d156c7c-dfe5-4aaf-a4e8-206a71114338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:57.531092 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1069/2141] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:30:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:30:57.540426 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f679ec-8f1e-4f49-b61d-96ee8d674592 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:57.541841 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-36f679ec-8f1e-4f49-b61d-96ee8d674592 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:57.600109 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f679ec-8f1e-4f49-b61d-96ee8d674592 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:57.606002 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d53203-b7f7-456d-abf6-b1d774d28184 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:57.606691 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d53203-b7f7-456d-abf6-b1d774d28184 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:57.606969 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d53203-b7f7-456d-abf6-b1d774d28184 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:57.607236 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d53203-b7f7-456d-abf6-b1d774d28184 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:57.618140 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d53203-b7f7-456d-abf6-b1d774d28184 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:57.619789 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d53203-b7f7-456d-abf6-b1d774d28184 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:57.630144 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1073/2142] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:30:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:30:58.032224 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d156c7c-dfe5-4aaf-a4e8-206a71114338 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:58.033937 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-2d156c7c-dfe5-4aaf-a4e8-206a71114338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:58.048124 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d156c7c-dfe5-4aaf-a4e8-206a71114338 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:58.053834 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12673d40-f5ca-4058-ab81-43262a96bfd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:58.054984 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12673d40-f5ca-4058-ab81-43262a96bfd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:58.055178 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12673d40-f5ca-4058-ab81-43262a96bfd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:58.055443 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12673d40-f5ca-4058-ab81-43262a96bfd9 admin admin] PATH_INFO: `/v3/projects/e9b5d2193ca54fdeaf28ec699a99fc26` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:58.056926 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12673d40-f5ca-4058-ab81-43262a96bfd9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9b5d2193ca54fdeaf28ec699a99fc26)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:58.059521 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12673d40-f5ca-4058-ab81-43262a96bfd9 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:58.158199 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-12673d40-f5ca-4058-ab81-43262a96bfd9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9b5d2193ca54fdeaf28ec699a99fc26', 'request_id': 'req-12673d40-f5ca-4058-ab81-43262a96bfd9'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:58.163678 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-12673d40-f5ca-4058-ab81-43262a96bfd9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9b5d2193ca54fdeaf28ec699a99fc26', 'request_id': 'req-12673d40-f5ca-4058-ab81-43262a96bfd9'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:58.227447 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-12673d40-f5ca-4058-ab81-43262a96bfd9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9b5d2193ca54fdeaf28ec699a99fc26', 'request_id': 'req-12673d40-f5ca-4058-ab81-43262a96bfd9'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:58.239721 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-12673d40-f5ca-4058-ab81-43262a96bfd9 admin admin] The token cache is being invalidate because project e9b5d2193ca54fdeaf28ec699a99fc26 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 17 19:30:58.240003 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-12673d40-f5ca-4058-ab81-43262a96bfd9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:58.240630 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-12673d40-f5ca-4058-ab81-43262a96bfd9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:58.242166 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1070/2143] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 17 19:30:58 2022] DELETE /identity/v3/projects/e9b5d2193ca54fdeaf28ec699a99fc26 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:30:58.247743 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668536a1-9dc7-49a9-a7c6-2401931074e2 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:58.248201 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668536a1-9dc7-49a9-a7c6-2401931074e2 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:58.248354 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668536a1-9dc7-49a9-a7c6-2401931074e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:58.266048 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-668536a1-9dc7-49a9-a7c6-2401931074e2 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:30:58.267248 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-668536a1-9dc7-49a9-a7c6-2401931074e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:58.326604 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1074/2144] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:30:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:30:58.337963 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef26923-1c33-491d-a8ff-9fb8989f07e2 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:58.337963 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef26923-1c33-491d-a8ff-9fb8989f07e2 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:58.337963 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef26923-1c33-491d-a8ff-9fb8989f07e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:58.354076 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-cef26923-1c33-491d-a8ff-9fb8989f07e2 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:30:58.355839 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-cef26923-1c33-491d-a8ff-9fb8989f07e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:58.363875 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1071/2145] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:30:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:30:58.368037 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668536a1-9dc7-49a9-a7c6-2401931074e2 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:58.377685 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668536a1-9dc7-49a9-a7c6-2401931074e2 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:58.382222 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf36588e-f738-42a4-bf8d-ff2f6a58a926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:58.382854 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf36588e-f738-42a4-bf8d-ff2f6a58a926 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:58.383128 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf36588e-f738-42a4-bf8d-ff2f6a58a926 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:58.383397 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf36588e-f738-42a4-bf8d-ff2f6a58a926 admin admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:58.384136 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf36588e-f738-42a4-bf8d-ff2f6a58a926 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:30:58.384437 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf36588e-f738-42a4-bf8d-ff2f6a58a926 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:58.386531 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf36588e-f738-42a4-bf8d-ff2f6a58a926 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:58.397618 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1075/2146] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 17 19:30:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:30:58.401578 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef26923-1c33-491d-a8ff-9fb8989f07e2 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:58.411072 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef26923-1c33-491d-a8ff-9fb8989f07e2 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:58.415803 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0911f69a-f1dd-4eb9-98e0-ddf44557ed76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:58.416487 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0911f69a-f1dd-4eb9-98e0-ddf44557ed76 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:58.416761 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0911f69a-f1dd-4eb9-98e0-ddf44557ed76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:58.417028 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0911f69a-f1dd-4eb9-98e0-ddf44557ed76 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:58.417751 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0911f69a-f1dd-4eb9-98e0-ddf44557ed76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:58.419669 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0911f69a-f1dd-4eb9-98e0-ddf44557ed76 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:58.490993 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1072/2147] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 17 19:30:58 2022] POST /identity/v3/projects => generated 373 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:30:58.495079 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf36588e-f738-42a4-bf8d-ff2f6a58a926 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:58.505715 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf36588e-f738-42a4-bf8d-ff2f6a58a926 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:58.565169 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3291ccd8-bdf1-4cbb-932e-aff774e89267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:58.565839 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3291ccd8-bdf1-4cbb-932e-aff774e89267 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:58.566122 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3291ccd8-bdf1-4cbb-932e-aff774e89267 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:58.566392 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3291ccd8-bdf1-4cbb-932e-aff774e89267 admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:58.567084 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3291ccd8-bdf1-4cbb-932e-aff774e89267 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:58.569029 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3291ccd8-bdf1-4cbb-932e-aff774e89267 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:58.691459 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1076/2148] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 17 19:30:58 2022] POST /identity/v3/users => generated 287 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:30:58.692776 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0911f69a-f1dd-4eb9-98e0-ddf44557ed76 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:58.703274 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0911f69a-f1dd-4eb9-98e0-ddf44557ed76 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:58.707811 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4524308d-3d17-4aa9-acd4-85357aa666dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:58.708455 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4524308d-3d17-4aa9-acd4-85357aa666dc admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:58.708734 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4524308d-3d17-4aa9-acd4-85357aa666dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:58.709005 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4524308d-3d17-4aa9-acd4-85357aa666dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:58.709724 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4524308d-3d17-4aa9-acd4-85357aa666dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:58.711318 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4524308d-3d17-4aa9-acd4-85357aa666dc admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:58.722037 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1073/2149] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:30:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:30:58.725796 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3291ccd8-bdf1-4cbb-932e-aff774e89267 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:58.735562 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3291ccd8-bdf1-4cbb-932e-aff774e89267 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:58.740129 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef7f123-c0fb-4e5d-893e-e878eeecd89d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:58.740766 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef7f123-c0fb-4e5d-893e-e878eeecd89d admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:58.741068 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef7f123-c0fb-4e5d-893e-e878eeecd89d admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:58.741316 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef7f123-c0fb-4e5d-893e-e878eeecd89d admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:58.742030 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef7f123-c0fb-4e5d-893e-e878eeecd89d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:58.743651 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef7f123-c0fb-4e5d-893e-e878eeecd89d admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:58.753887 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1077/2150] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:30:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 17 19:30:58.757798 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4524308d-3d17-4aa9-acd4-85357aa666dc admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:58.768516 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4524308d-3d17-4aa9-acd4-85357aa666dc admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:58.773321 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c778838-1f38-4c31-9d04-00fed4c1bdc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:58.774221 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c778838-1f38-4c31-9d04-00fed4c1bdc1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:58.774546 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c778838-1f38-4c31-9d04-00fed4c1bdc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:58.775216 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c778838-1f38-4c31-9d04-00fed4c1bdc1 admin admin] PATH_INFO: `/v3/projects/e005e3c10a1441679b3b00340abd76cb/users/699ab697b7e0421f93a3662a1c6b66b5/roles/114e6a16e63447d1924707ace5aace25` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:58.790589 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c778838-1f38-4c31-9d04-00fed4c1bdc1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e005e3c10a1441679b3b00340abd76cb, user_id=699ab697b7e0421f93a3662a1c6b66b5, role_id=114e6a16e63447d1924707ace5aace25)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:58.793201 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c778838-1f38-4c31-9d04-00fed4c1bdc1 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:58.919279 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1074/2151] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:30:58 2022] PUT /identity/v3/projects/e005e3c10a1441679b3b00340abd76cb/users/699ab697b7e0421f93a3662a1c6b66b5/roles/114e6a16e63447d1924707ace5aace25 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:30:58.929496 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87d5aa3-cfcc-4b43-8dd2-8cbbb2286f9a None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:58.929785 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87d5aa3-cfcc-4b43-8dd2-8cbbb2286f9a None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:58.930063 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87d5aa3-cfcc-4b43-8dd2-8cbbb2286f9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:58.956680 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-b87d5aa3-cfcc-4b43-8dd2-8cbbb2286f9a None None] MFA Rules not processed for user `699ab697b7e0421f93a3662a1c6b66b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:30:58.957877 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-b87d5aa3-cfcc-4b43-8dd2-8cbbb2286f9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:59.019720 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1078/2152] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:30:58 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:30:59.030934 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c778838-1f38-4c31-9d04-00fed4c1bdc1 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:59.032632 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-4c778838-1f38-4c31-9d04-00fed4c1bdc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:59.088120 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c778838-1f38-4c31-9d04-00fed4c1bdc1 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:59.091898 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb4179e-5ea6-4df7-90cb-856abaf9f899 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd05409b38ae346458abfe41ebda1c59e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:59.092528 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb4179e-5ea6-4df7-90cb-856abaf9f899 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:59.092804 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb4179e-5ea6-4df7-90cb-856abaf9f899 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:59.093075 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb4179e-5ea6-4df7-90cb-856abaf9f899 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:59.101700 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb4179e-5ea6-4df7-90cb-856abaf9f899 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:59.103127 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb4179e-5ea6-4df7-90cb-856abaf9f899 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:59.112552 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1075/2153] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:30:59 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:30:59.137210 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87d5aa3-cfcc-4b43-8dd2-8cbbb2286f9a None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:59.195422 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87d5aa3-cfcc-4b43-8dd2-8cbbb2286f9a None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:59.200048 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8339da4c-df2f-4da4-b70d-649fd4868a12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:59.200895 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8339da4c-df2f-4da4-b70d-649fd4868a12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:59.201170 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8339da4c-df2f-4da4-b70d-649fd4868a12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:59.201439 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8339da4c-df2f-4da4-b70d-649fd4868a12 admin admin] PATH_INFO: `/v3/users/699ab697b7e0421f93a3662a1c6b66b5` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:59.202585 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8339da4c-df2f-4da4-b70d-649fd4868a12 admin admin] RBAC: Authorizing `identity:delete_user(user_id=699ab697b7e0421f93a3662a1c6b66b5)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:59.204526 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8339da4c-df2f-4da4-b70d-649fd4868a12 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:59.395789 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-8339da4c-df2f-4da4-b70d-649fd4868a12 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '699ab697b7e0421f93a3662a1c6b66b5'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:59.403342 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-8339da4c-df2f-4da4-b70d-649fd4868a12 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '699ab697b7e0421f93a3662a1c6b66b5'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:59.418373 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-8339da4c-df2f-4da4-b70d-649fd4868a12 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '699ab697b7e0421f93a3662a1c6b66b5'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:59.419106 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-8339da4c-df2f-4da4-b70d-649fd4868a12 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '699ab697b7e0421f93a3662a1c6b66b5'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:59.419778 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-8339da4c-df2f-4da4-b70d-649fd4868a12 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '699ab697b7e0421f93a3662a1c6b66b5'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:30:59.528477 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1079/2154] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 17 19:30:59 2022] DELETE /identity/v3/users/699ab697b7e0421f93a3662a1c6b66b5 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:30:59.533101 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3189ae-b95e-4278-9ff9-8855e9d82732 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:59.533614 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3189ae-b95e-4278-9ff9-8855e9d82732 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:59.533709 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3189ae-b95e-4278-9ff9-8855e9d82732 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:59.552061 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-3e3189ae-b95e-4278-9ff9-8855e9d82732 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:30:59.553291 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-3e3189ae-b95e-4278-9ff9-8855e9d82732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:59.613853 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1076/2155] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:30:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:30:59.624740 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8339da4c-df2f-4da4-b70d-649fd4868a12 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:59.626171 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-8339da4c-df2f-4da4-b70d-649fd4868a12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:59.685367 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8339da4c-df2f-4da4-b70d-649fd4868a12 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:59.689136 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eb7d32-072a-4010-9aec-f324afb8f852 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:59.689746 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eb7d32-072a-4010-9aec-f324afb8f852 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:59.690021 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eb7d32-072a-4010-9aec-f324afb8f852 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:59.690301 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eb7d32-072a-4010-9aec-f324afb8f852 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:59.702319 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4eb7d32-072a-4010-9aec-f324afb8f852 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:30:59.702319 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4eb7d32-072a-4010-9aec-f324afb8f852 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:30:59.710566 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1080/2156] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:30:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:30:59.976043 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3189ae-b95e-4278-9ff9-8855e9d82732 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:30:59.977726 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-3e3189ae-b95e-4278-9ff9-8855e9d82732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:30:59.991665 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3189ae-b95e-4278-9ff9-8855e9d82732 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:30:59.996676 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dc2897-0c05-4386-958c-7127ce85031f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:30:59.997614 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dc2897-0c05-4386-958c-7127ce85031f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:30:59.997964 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dc2897-0c05-4386-958c-7127ce85031f admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:30:59.998245 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dc2897-0c05-4386-958c-7127ce85031f admin admin] PATH_INFO: `/v3/users/b209201893dc4b27a7f0d9423a30899d` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:30:59.999640 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dc2897-0c05-4386-958c-7127ce85031f admin admin] RBAC: Authorizing `identity:delete_user(user_id=b209201893dc4b27a7f0d9423a30899d)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:00.001784 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dc2897-0c05-4386-958c-7127ce85031f admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:00.181196 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eb7d32-072a-4010-9aec-f324afb8f852 service neutron] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:00.182865 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-c4eb7d32-072a-4010-9aec-f324afb8f852 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:00.198569 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eb7d32-072a-4010-9aec-f324afb8f852 service neutron] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:00.204231 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c748aafa-f23b-4e10-bfeb-82dec3e38d77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:00.212965 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c748aafa-f23b-4e10-bfeb-82dec3e38d77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:00.212965 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c748aafa-f23b-4e10-bfeb-82dec3e38d77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:00.212965 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c748aafa-f23b-4e10-bfeb-82dec3e38d77 admin admin] PATH_INFO: `/v3/projects/e005e3c10a1441679b3b00340abd76cb` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:00.212965 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c748aafa-f23b-4e10-bfeb-82dec3e38d77 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e005e3c10a1441679b3b00340abd76cb)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:00.212965 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c748aafa-f23b-4e10-bfeb-82dec3e38d77 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:00.220930 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-32dc2897-0c05-4386-958c-7127ce85031f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b209201893dc4b27a7f0d9423a30899d'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:00.222017 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-32dc2897-0c05-4386-958c-7127ce85031f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b209201893dc4b27a7f0d9423a30899d'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:00.235030 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-32dc2897-0c05-4386-958c-7127ce85031f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b209201893dc4b27a7f0d9423a30899d'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:00.367832 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-32dc2897-0c05-4386-958c-7127ce85031f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b209201893dc4b27a7f0d9423a30899d'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:00.370444 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-c748aafa-f23b-4e10-bfeb-82dec3e38d77 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e005e3c10a1441679b3b00340abd76cb', 'request_id': 'req-c748aafa-f23b-4e10-bfeb-82dec3e38d77'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:00.375283 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-32dc2897-0c05-4386-958c-7127ce85031f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b209201893dc4b27a7f0d9423a30899d'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:00.378228 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-c748aafa-f23b-4e10-bfeb-82dec3e38d77 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e005e3c10a1441679b3b00340abd76cb', 'request_id': 'req-c748aafa-f23b-4e10-bfeb-82dec3e38d77'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:00.381289 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1077/2157] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 17 19:30:59 2022] DELETE /identity/v3/users/b209201893dc4b27a7f0d9423a30899d => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:31:00.383793 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-c748aafa-f23b-4e10-bfeb-82dec3e38d77 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e005e3c10a1441679b3b00340abd76cb', 'request_id': 'req-c748aafa-f23b-4e10-bfeb-82dec3e38d77'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:00.385683 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7eccfc-7f1a-4fda-b353-6b2e23589a9f None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:00.386924 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7eccfc-7f1a-4fda-b353-6b2e23589a9f None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:00.387260 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7eccfc-7f1a-4fda-b353-6b2e23589a9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:00.406099 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-7f7eccfc-7f1a-4fda-b353-6b2e23589a9f None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:31:00.407421 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-7f7eccfc-7f1a-4fda-b353-6b2e23589a9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:00.469105 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-c748aafa-f23b-4e10-bfeb-82dec3e38d77 admin admin] The token cache is being invalidate because project e005e3c10a1441679b3b00340abd76cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 17 19:31:00.469666 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-c748aafa-f23b-4e10-bfeb-82dec3e38d77 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:00.470762 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-c748aafa-f23b-4e10-bfeb-82dec3e38d77 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:00.473067 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1081/2158] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 17 19:31:00 2022] DELETE /identity/v3/projects/e005e3c10a1441679b3b00340abd76cb => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 17 19:31:00.479751 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b785e14-be86-4832-bb9d-63870d80adaf None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:00.480296 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b785e14-be86-4832-bb9d-63870d80adaf None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:00.480689 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b785e14-be86-4832-bb9d-63870d80adaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:00.486866 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1078/2159] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:31:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:31:00.501018 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7eccfc-7f1a-4fda-b353-6b2e23589a9f None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:00.502404 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-7f7eccfc-7f1a-4fda-b353-6b2e23589a9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:00.507800 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-5b785e14-be86-4832-bb9d-63870d80adaf None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:31:00.508979 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-5b785e14-be86-4832-bb9d-63870d80adaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:00.532753 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1082/2160] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:31:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:31:00.534408 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b338fe4-20dc-4fc5-a682-114ab36a47a2 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:00.534776 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b338fe4-20dc-4fc5-a682-114ab36a47a2 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:00.535135 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b338fe4-20dc-4fc5-a682-114ab36a47a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:00.554187 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-5b338fe4-20dc-4fc5-a682-114ab36a47a2 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:31:00.555598 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-5b338fe4-20dc-4fc5-a682-114ab36a47a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:00.564577 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1083/2161] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:31:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:31:00.568926 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b338fe4-20dc-4fc5-a682-114ab36a47a2 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:00.579101 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b338fe4-20dc-4fc5-a682-114ab36a47a2 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:00.589815 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff21ebc-c8ff-48f8-8881-45c2c4179682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:00.589815 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff21ebc-c8ff-48f8-8881-45c2c4179682 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:00.589815 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff21ebc-c8ff-48f8-8881-45c2c4179682 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:00.589815 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff21ebc-c8ff-48f8-8881-45c2c4179682 admin admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:00.590243 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7eccfc-7f1a-4fda-b353-6b2e23589a9f None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:00.590732 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff21ebc-c8ff-48f8-8881-45c2c4179682 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:31:00.590732 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff21ebc-c8ff-48f8-8881-45c2c4179682 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:00.590732 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff21ebc-c8ff-48f8-8881-45c2c4179682 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:00.593262 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894a3c8f-c597-4deb-908d-6d2b1501b9de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:00.594107 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894a3c8f-c597-4deb-908d-6d2b1501b9de service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:00.594668 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894a3c8f-c597-4deb-908d-6d2b1501b9de service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:00.595119 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894a3c8f-c597-4deb-908d-6d2b1501b9de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:00.603619 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1084/2162] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 17 19:31:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:31:00.610295 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff21ebc-c8ff-48f8-8881-45c2c4179682 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:00.610424 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894a3c8f-c597-4deb-908d-6d2b1501b9de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:00.610424 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894a3c8f-c597-4deb-908d-6d2b1501b9de service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:00.623028 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff21ebc-c8ff-48f8-8881-45c2c4179682 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:00.624043 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1079/2163] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:31:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 17 19:31:00.629797 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca9673d-1bb9-44aa-8a68-0d4ec0888dee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:00.631344 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca9673d-1bb9-44aa-8a68-0d4ec0888dee admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:00.631344 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca9673d-1bb9-44aa-8a68-0d4ec0888dee admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:00.631344 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca9673d-1bb9-44aa-8a68-0d4ec0888dee admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:00.631851 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca9673d-1bb9-44aa-8a68-0d4ec0888dee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:00.634639 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca9673d-1bb9-44aa-8a68-0d4ec0888dee admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:00.746286 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1085/2164] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 17 19:31:00 2022] POST /identity/v3/projects => generated 373 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:31:00.750102 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894a3c8f-c597-4deb-908d-6d2b1501b9de service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:00.760088 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894a3c8f-c597-4deb-908d-6d2b1501b9de service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:00.842924 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b98b46-ec55-4b49-a997-292ae3d16307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:00.843651 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b98b46-ec55-4b49-a997-292ae3d16307 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:00.843982 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b98b46-ec55-4b49-a997-292ae3d16307 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:00.844740 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b98b46-ec55-4b49-a997-292ae3d16307 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:00.845448 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b98b46-ec55-4b49-a997-292ae3d16307 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:00.848219 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b98b46-ec55-4b49-a997-292ae3d16307 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:00.952240 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1080/2165] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 17 19:31:00 2022] POST /identity/v3/users => generated 287 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:31:00.953577 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca9673d-1bb9-44aa-8a68-0d4ec0888dee admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:00.967364 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca9673d-1bb9-44aa-8a68-0d4ec0888dee admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:00.972575 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf8ffb5-0190-4ee2-a396-f4460ca5619f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:00.973281 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf8ffb5-0190-4ee2-a396-f4460ca5619f admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:00.973559 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf8ffb5-0190-4ee2-a396-f4460ca5619f admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:00.973827 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf8ffb5-0190-4ee2-a396-f4460ca5619f admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:00.974751 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf8ffb5-0190-4ee2-a396-f4460ca5619f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:00.976404 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf8ffb5-0190-4ee2-a396-f4460ca5619f admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:00.987627 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1086/2166] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:31:00 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:31:00.991394 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b98b46-ec55-4b49-a997-292ae3d16307 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:01.001520 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b98b46-ec55-4b49-a997-292ae3d16307 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:01.006254 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3271af-512c-451a-875e-847faeec6432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:01.006925 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3271af-512c-451a-875e-847faeec6432 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:01.007211 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3271af-512c-451a-875e-847faeec6432 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:01.007506 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3271af-512c-451a-875e-847faeec6432 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:01.008244 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3271af-512c-451a-875e-847faeec6432 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:01.009964 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3271af-512c-451a-875e-847faeec6432 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:01.026144 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1081/2167] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:31:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:31:01.027916 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf8ffb5-0190-4ee2-a396-f4460ca5619f admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:01.037918 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf8ffb5-0190-4ee2-a396-f4460ca5619f admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:01.043380 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3563399c-b54a-467a-80a1-a4487161e242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:01.044412 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3563399c-b54a-467a-80a1-a4487161e242 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:01.044661 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3563399c-b54a-467a-80a1-a4487161e242 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:01.044934 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3563399c-b54a-467a-80a1-a4487161e242 admin admin] PATH_INFO: `/v3/projects/14a6f2a77ec74b60815121fc444718c6/users/3e10bbd1be814c9f8216bf64b4128e54/roles/114e6a16e63447d1924707ace5aace25` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:01.059260 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3563399c-b54a-467a-80a1-a4487161e242 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14a6f2a77ec74b60815121fc444718c6, user_id=3e10bbd1be814c9f8216bf64b4128e54, role_id=114e6a16e63447d1924707ace5aace25)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:01.062054 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3563399c-b54a-467a-80a1-a4487161e242 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:01.098252 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3271af-512c-451a-875e-847faeec6432 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:01.100173 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-9d3271af-512c-451a-875e-847faeec6432 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:01.117119 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3271af-512c-451a-875e-847faeec6432 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:01.127314 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4dfe2e-ad84-46db-9a2c-dd23f6f2fcb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:01.127314 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4dfe2e-ad84-46db-9a2c-dd23f6f2fcb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:01.127314 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4dfe2e-ad84-46db-9a2c-dd23f6f2fcb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:01.127314 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4dfe2e-ad84-46db-9a2c-dd23f6f2fcb9 admin admin] PATH_INFO: `/v3/projects/8553cad4ce8a4fbd8d87d11f21f53872` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:01.128237 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4dfe2e-ad84-46db-9a2c-dd23f6f2fcb9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8553cad4ce8a4fbd8d87d11f21f53872)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:01.130036 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4dfe2e-ad84-46db-9a2c-dd23f6f2fcb9 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:01.157350 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1087/2168] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:31:01 2022] PUT /identity/v3/projects/14a6f2a77ec74b60815121fc444718c6/users/3e10bbd1be814c9f8216bf64b4128e54/roles/114e6a16e63447d1924707ace5aace25 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:31:01.168625 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b274ed-9f63-4610-8ef2-65b762d1db27 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:01.168996 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b274ed-9f63-4610-8ef2-65b762d1db27 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:01.169346 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b274ed-9f63-4610-8ef2-65b762d1db27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:01.197539 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-c0b274ed-9f63-4610-8ef2-65b762d1db27 None None] MFA Rules not processed for user `3e10bbd1be814c9f8216bf64b4128e54`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:31:01.198783 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-c0b274ed-9f63-4610-8ef2-65b762d1db27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:01.279424 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1088/2169] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:31:01 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 17 19:31:01.291090 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b274ed-9f63-4610-8ef2-65b762d1db27 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:01.293088 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-c0b274ed-9f63-4610-8ef2-65b762d1db27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:01.329060 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0f4dfe2e-ad84-46db-9a2c-dd23f6f2fcb9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8553cad4ce8a4fbd8d87d11f21f53872', 'request_id': 'req-0f4dfe2e-ad84-46db-9a2c-dd23f6f2fcb9'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:01.335735 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0f4dfe2e-ad84-46db-9a2c-dd23f6f2fcb9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8553cad4ce8a4fbd8d87d11f21f53872', 'request_id': 'req-0f4dfe2e-ad84-46db-9a2c-dd23f6f2fcb9'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:01.350825 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b274ed-9f63-4610-8ef2-65b762d1db27 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:01.401712 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1238502-0249-4dd9-82f7-1a7e222ab4d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd05409b38ae346458abfe41ebda1c59e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:01.402306 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1238502-0249-4dd9-82f7-1a7e222ab4d5 service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:01.402604 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1238502-0249-4dd9-82f7-1a7e222ab4d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:01.402887 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1238502-0249-4dd9-82f7-1a7e222ab4d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:01.410264 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0f4dfe2e-ad84-46db-9a2c-dd23f6f2fcb9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8553cad4ce8a4fbd8d87d11f21f53872', 'request_id': 'req-0f4dfe2e-ad84-46db-9a2c-dd23f6f2fcb9'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:01.448687 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0f4dfe2e-ad84-46db-9a2c-dd23f6f2fcb9 admin admin] The token cache is being invalidate because project 8553cad4ce8a4fbd8d87d11f21f53872 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 17 19:31:01.448687 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0f4dfe2e-ad84-46db-9a2c-dd23f6f2fcb9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:01.448687 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0f4dfe2e-ad84-46db-9a2c-dd23f6f2fcb9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:01.450738 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1082/2170] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 17 19:31:01 2022] DELETE /identity/v3/projects/8553cad4ce8a4fbd8d87d11f21f53872 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:31:01.455623 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08819c29-b8d3-44e9-8306-74df9652660b None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:01.455922 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08819c29-b8d3-44e9-8306-74df9652660b None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:01.456914 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08819c29-b8d3-44e9-8306-74df9652660b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:01.466978 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1238502-0249-4dd9-82f7-1a7e222ab4d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:01.468639 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1238502-0249-4dd9-82f7-1a7e222ab4d5 service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:01.475863 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-08819c29-b8d3-44e9-8306-74df9652660b None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:31:01.477049 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-08819c29-b8d3-44e9-8306-74df9652660b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:01.478531 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1089/2171] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:31:01 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 17 19:31:01.561331 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1083/2172] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:31:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 17 19:31:01.568243 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73451329-dce7-4502-848f-0ab04bc275c0 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:01.568927 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73451329-dce7-4502-848f-0ab04bc275c0 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:01.569407 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73451329-dce7-4502-848f-0ab04bc275c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:01.593168 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-73451329-dce7-4502-848f-0ab04bc275c0 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:31:01.594389 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-73451329-dce7-4502-848f-0ab04bc275c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:01.602880 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1090/2173] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:31:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:31:01.608069 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08819c29-b8d3-44e9-8306-74df9652660b None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:01.617841 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08819c29-b8d3-44e9-8306-74df9652660b None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:01.622530 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87f3ed6-7247-4440-be6a-5597a49d21e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:01.623185 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87f3ed6-7247-4440-be6a-5597a49d21e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:01.623594 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87f3ed6-7247-4440-be6a-5597a49d21e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:01.623908 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87f3ed6-7247-4440-be6a-5597a49d21e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:01.624623 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87f3ed6-7247-4440-be6a-5597a49d21e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:31:01.624921 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87f3ed6-7247-4440-be6a-5597a49d21e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:01.626510 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87f3ed6-7247-4440-be6a-5597a49d21e5 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:01.639839 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1084/2174] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 17 19:31:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:31:01.644189 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73451329-dce7-4502-848f-0ab04bc275c0 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:01.654806 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73451329-dce7-4502-848f-0ab04bc275c0 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:01.659954 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4456652d-fc23-4c91-bcb5-cefcb4db4e78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:01.660996 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4456652d-fc23-4c91-bcb5-cefcb4db4e78 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:01.661303 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4456652d-fc23-4c91-bcb5-cefcb4db4e78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:01.665669 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4456652d-fc23-4c91-bcb5-cefcb4db4e78 admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:01.665669 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4456652d-fc23-4c91-bcb5-cefcb4db4e78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:01.665669 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4456652d-fc23-4c91-bcb5-cefcb4db4e78 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:01.767100 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1091/2175] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 17 19:31:01 2022] POST /identity/v3/projects => generated 405 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:31:01.771293 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87f3ed6-7247-4440-be6a-5597a49d21e5 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:01.782493 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87f3ed6-7247-4440-be6a-5597a49d21e5 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:01.844266 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8edb374-942c-47cd-aa4a-994e9b33d37f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:01.845216 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8edb374-942c-47cd-aa4a-994e9b33d37f admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:01.845537 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8edb374-942c-47cd-aa4a-994e9b33d37f admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:01.845847 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8edb374-942c-47cd-aa4a-994e9b33d37f admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:01.846963 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8edb374-942c-47cd-aa4a-994e9b33d37f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:01.849255 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8edb374-942c-47cd-aa4a-994e9b33d37f admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:01.920604 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1085/2176] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 17 19:31:01 2022] POST /identity/v3/users => generated 303 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:31:01.923712 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4456652d-fc23-4c91-bcb5-cefcb4db4e78 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:01.933971 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4456652d-fc23-4c91-bcb5-cefcb4db4e78 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:01.938737 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffb81ec-0981-463c-99f3-12d40989763f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:01.941047 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffb81ec-0981-463c-99f3-12d40989763f admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:01.941047 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffb81ec-0981-463c-99f3-12d40989763f admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:01.941047 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffb81ec-0981-463c-99f3-12d40989763f admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:01.941047 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffb81ec-0981-463c-99f3-12d40989763f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:01.942239 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffb81ec-0981-463c-99f3-12d40989763f admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:01.955687 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1092/2177] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:31:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:31:01.960131 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8edb374-942c-47cd-aa4a-994e9b33d37f admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:01.971644 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8edb374-942c-47cd-aa4a-994e9b33d37f admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:01.976769 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba033df1-7449-4f09-8573-ce568817bdcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:01.977431 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba033df1-7449-4f09-8573-ce568817bdcb admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:01.977708 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba033df1-7449-4f09-8573-ce568817bdcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:01.977979 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba033df1-7449-4f09-8573-ce568817bdcb admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:01.978709 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba033df1-7449-4f09-8573-ce568817bdcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:01.980467 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba033df1-7449-4f09-8573-ce568817bdcb admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:01.990390 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1086/2178] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:31:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 17 19:31:01.994717 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffb81ec-0981-463c-99f3-12d40989763f admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:02.004653 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffb81ec-0981-463c-99f3-12d40989763f admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:02.009264 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e137be8-7b12-456c-94b9-61455d71221d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:02.010122 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e137be8-7b12-456c-94b9-61455d71221d admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:02.010410 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e137be8-7b12-456c-94b9-61455d71221d admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:02.010700 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e137be8-7b12-456c-94b9-61455d71221d admin admin] PATH_INFO: `/v3/projects/b77623616a094df29ee7a94f74d7bfee/users/a9c653d4417b447485a58935bf478092/roles/114e6a16e63447d1924707ace5aace25` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:02.027329 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e137be8-7b12-456c-94b9-61455d71221d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b77623616a094df29ee7a94f74d7bfee, user_id=a9c653d4417b447485a58935bf478092, role_id=114e6a16e63447d1924707ace5aace25)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:02.030081 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e137be8-7b12-456c-94b9-61455d71221d admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:02.093189 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1093/2179] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:31:01 2022] PUT /identity/v3/projects/b77623616a094df29ee7a94f74d7bfee/users/a9c653d4417b447485a58935bf478092/roles/114e6a16e63447d1924707ace5aace25 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:31:02.104322 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5cb6d3-cc7a-47ce-9451-ed42bf351b8f None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:02.104662 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5cb6d3-cc7a-47ce-9451-ed42bf351b8f None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:02.105166 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5cb6d3-cc7a-47ce-9451-ed42bf351b8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:02.134889 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-9f5cb6d3-cc7a-47ce-9451-ed42bf351b8f None None] MFA Rules not processed for user `a9c653d4417b447485a58935bf478092`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:31:02.135272 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-9f5cb6d3-cc7a-47ce-9451-ed42bf351b8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:02.209796 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1087/2180] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:31:02 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:31:02.215323 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e137be8-7b12-456c-94b9-61455d71221d admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:02.293474 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e137be8-7b12-456c-94b9-61455d71221d admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:02.298416 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10377a11-29eb-4a46-91ed-60fa42fb2fca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:02.299070 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10377a11-29eb-4a46-91ed-60fa42fb2fca admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:02.299348 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10377a11-29eb-4a46-91ed-60fa42fb2fca admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:02.299655 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10377a11-29eb-4a46-91ed-60fa42fb2fca admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:02.300439 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10377a11-29eb-4a46-91ed-60fa42fb2fca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:02.302478 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10377a11-29eb-4a46-91ed-60fa42fb2fca admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:02.423557 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1094/2181] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 17 19:31:02 2022] POST /identity/v3/projects => generated 405 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 17 19:31:02.427776 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5cb6d3-cc7a-47ce-9451-ed42bf351b8f None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:02.439345 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5cb6d3-cc7a-47ce-9451-ed42bf351b8f None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:02.499197 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a98243-1326-4779-96bf-f26b14611c67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:02.499707 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a98243-1326-4779-96bf-f26b14611c67 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:02.500011 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a98243-1326-4779-96bf-f26b14611c67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:02.500287 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a98243-1326-4779-96bf-f26b14611c67 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:02.501036 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a98243-1326-4779-96bf-f26b14611c67 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:02.503119 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a98243-1326-4779-96bf-f26b14611c67 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:02.605451 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1088/2182] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 17 19:31:02 2022] POST /identity/v3/users => generated 309 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:31:02.609287 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10377a11-29eb-4a46-91ed-60fa42fb2fca admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:02.620846 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10377a11-29eb-4a46-91ed-60fa42fb2fca admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:02.625914 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461f3cef-a42b-43e7-86ec-c9b3039fecf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:02.626619 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461f3cef-a42b-43e7-86ec-c9b3039fecf8 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:02.626958 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461f3cef-a42b-43e7-86ec-c9b3039fecf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:02.627241 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461f3cef-a42b-43e7-86ec-c9b3039fecf8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:02.628052 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461f3cef-a42b-43e7-86ec-c9b3039fecf8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:02.629695 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461f3cef-a42b-43e7-86ec-c9b3039fecf8 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:02.641314 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1095/2183] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:31:02 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:31:02.645912 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a98243-1326-4779-96bf-f26b14611c67 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:02.657108 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a98243-1326-4779-96bf-f26b14611c67 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:02.662030 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780bc46f-4ba4-42a8-ba34-20a547b5118d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:02.662904 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780bc46f-4ba4-42a8-ba34-20a547b5118d admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:02.663175 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780bc46f-4ba4-42a8-ba34-20a547b5118d admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:02.663538 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780bc46f-4ba4-42a8-ba34-20a547b5118d admin admin] PATH_INFO: `/v3/projects/b1b76b871deb4435a4b3ee44b488e05e/users/3fa02ec5338442fcb560461b455e3346/roles/60af5088793e4379b88821a86552a344` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:02.678717 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780bc46f-4ba4-42a8-ba34-20a547b5118d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1b76b871deb4435a4b3ee44b488e05e, user_id=3fa02ec5338442fcb560461b455e3346, role_id=60af5088793e4379b88821a86552a344)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:02.681558 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780bc46f-4ba4-42a8-ba34-20a547b5118d admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:02.784340 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1089/2184] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:31:02 2022] PUT /identity/v3/projects/b1b76b871deb4435a4b3ee44b488e05e/users/3fa02ec5338442fcb560461b455e3346/roles/60af5088793e4379b88821a86552a344 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:31:02.788721 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461f3cef-a42b-43e7-86ec-c9b3039fecf8 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:02.798772 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461f3cef-a42b-43e7-86ec-c9b3039fecf8 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:02.857072 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dc36c2-e560-49da-9097-5b7740e27872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:02.857764 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dc36c2-e560-49da-9097-5b7740e27872 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:02.858078 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dc36c2-e560-49da-9097-5b7740e27872 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:02.858370 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dc36c2-e560-49da-9097-5b7740e27872 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:02.859138 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dc36c2-e560-49da-9097-5b7740e27872 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:02.860941 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dc36c2-e560-49da-9097-5b7740e27872 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:02.872670 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1096/2185] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:31:02 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:31:02.877071 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780bc46f-4ba4-42a8-ba34-20a547b5118d admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:02.888063 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780bc46f-4ba4-42a8-ba34-20a547b5118d admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:02.893138 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b31fcb1-c292-4486-a4a0-af4cf8f24925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:02.894421 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b31fcb1-c292-4486-a4a0-af4cf8f24925 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:02.894872 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b31fcb1-c292-4486-a4a0-af4cf8f24925 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:02.895106 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b31fcb1-c292-4486-a4a0-af4cf8f24925 admin admin] PATH_INFO: `/v3/projects/b1b76b871deb4435a4b3ee44b488e05e/users/3fa02ec5338442fcb560461b455e3346/roles/60af5088793e4379b88821a86552a344` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:02.897433 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b31fcb1-c292-4486-a4a0-af4cf8f24925 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1b76b871deb4435a4b3ee44b488e05e, user_id=3fa02ec5338442fcb560461b455e3346, role_id=60af5088793e4379b88821a86552a344)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:02.901249 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b31fcb1-c292-4486-a4a0-af4cf8f24925 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:02.913833 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1090/2186] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:31:02 2022] PUT /identity/v3/projects/b1b76b871deb4435a4b3ee44b488e05e/users/3fa02ec5338442fcb560461b455e3346/roles/60af5088793e4379b88821a86552a344 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:31:02.926486 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c99b5d6-1291-456a-bb97-7d59067f59c4 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:02.926805 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c99b5d6-1291-456a-bb97-7d59067f59c4 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:02.927105 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c99b5d6-1291-456a-bb97-7d59067f59c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:02.967948 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-1c99b5d6-1291-456a-bb97-7d59067f59c4 None None] MFA Rules not processed for user `3fa02ec5338442fcb560461b455e3346`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:31:02.969438 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-1c99b5d6-1291-456a-bb97-7d59067f59c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:03.092514 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1097/2187] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:31:02 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:31:03.103859 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b31fcb1-c292-4486-a4a0-af4cf8f24925 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:03.105564 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-6b31fcb1-c292-4486-a4a0-af4cf8f24925 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:03.173561 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b31fcb1-c292-4486-a4a0-af4cf8f24925 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:03.190031 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f77d59a-1018-4d77-a8dd-c65a303025a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd05409b38ae346458abfe41ebda1c59e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:03.190852 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f77d59a-1018-4d77-a8dd-c65a303025a2 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:03.191149 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f77d59a-1018-4d77-a8dd-c65a303025a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:03.191430 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f77d59a-1018-4d77-a8dd-c65a303025a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:03.218763 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f77d59a-1018-4d77-a8dd-c65a303025a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:03.218763 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f77d59a-1018-4d77-a8dd-c65a303025a2 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:03.222185 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1091/2188] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:31:03 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:31:03.425045 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c99b5d6-1291-456a-bb97-7d59067f59c4 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:03.488734 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c99b5d6-1291-456a-bb97-7d59067f59c4 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:03.493479 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0594628a-ac3f-4651-bda6-0b16964a6a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:03.494375 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0594628a-ac3f-4651-bda6-0b16964a6a2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:03.494648 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0594628a-ac3f-4651-bda6-0b16964a6a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:03.494915 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0594628a-ac3f-4651-bda6-0b16964a6a2a admin admin] PATH_INFO: `/v3/users/a9c653d4417b447485a58935bf478092` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:03.497019 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0594628a-ac3f-4651-bda6-0b16964a6a2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9c653d4417b447485a58935bf478092)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:03.499540 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0594628a-ac3f-4651-bda6-0b16964a6a2a admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:03.705820 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-0594628a-ac3f-4651-bda6-0b16964a6a2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9c653d4417b447485a58935bf478092'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:03.716428 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-0594628a-ac3f-4651-bda6-0b16964a6a2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9c653d4417b447485a58935bf478092'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:03.739672 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-0594628a-ac3f-4651-bda6-0b16964a6a2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9c653d4417b447485a58935bf478092'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:03.739672 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-0594628a-ac3f-4651-bda6-0b16964a6a2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9c653d4417b447485a58935bf478092'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:03.739672 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-0594628a-ac3f-4651-bda6-0b16964a6a2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9c653d4417b447485a58935bf478092'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:03.836683 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1098/2189] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 17 19:31:03 2022] DELETE /identity/v3/users/a9c653d4417b447485a58935bf478092 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:31:03.841790 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e08ca9-5fb6-4d42-b6fc-f929e8d9f4ea None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:03.842281 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e08ca9-5fb6-4d42-b6fc-f929e8d9f4ea None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:03.842383 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e08ca9-5fb6-4d42-b6fc-f929e8d9f4ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:03.861584 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-01e08ca9-5fb6-4d42-b6fc-f929e8d9f4ea None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:31:03.862909 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-01e08ca9-5fb6-4d42-b6fc-f929e8d9f4ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:03.935470 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1092/2190] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:31:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:31:03.948487 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0594628a-ac3f-4651-bda6-0b16964a6a2a admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:03.949870 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-0594628a-ac3f-4651-bda6-0b16964a6a2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:04.011271 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0594628a-ac3f-4651-bda6-0b16964a6a2a admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:04.015207 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6610faaa-f92f-487a-98ac-ea41ca9449d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:04.015880 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6610faaa-f92f-487a-98ac-ea41ca9449d1 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:04.016157 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6610faaa-f92f-487a-98ac-ea41ca9449d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:04.016459 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6610faaa-f92f-487a-98ac-ea41ca9449d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:04.029634 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6610faaa-f92f-487a-98ac-ea41ca9449d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:04.032021 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6610faaa-f92f-487a-98ac-ea41ca9449d1 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:04.041669 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1099/2191] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:31:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:31:04.572163 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e08ca9-5fb6-4d42-b6fc-f929e8d9f4ea None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:04.573744 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-01e08ca9-5fb6-4d42-b6fc-f929e8d9f4ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:04.587370 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e08ca9-5fb6-4d42-b6fc-f929e8d9f4ea None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:04.592312 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a58e9a1-4e91-4ff0-94bf-bb31eaa3e897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:04.594563 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a58e9a1-4e91-4ff0-94bf-bb31eaa3e897 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:04.594563 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a58e9a1-4e91-4ff0-94bf-bb31eaa3e897 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:04.594563 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a58e9a1-4e91-4ff0-94bf-bb31eaa3e897 admin admin] PATH_INFO: `/v3/projects/b77623616a094df29ee7a94f74d7bfee` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:04.594937 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a58e9a1-4e91-4ff0-94bf-bb31eaa3e897 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b77623616a094df29ee7a94f74d7bfee)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:04.597147 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a58e9a1-4e91-4ff0-94bf-bb31eaa3e897 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:04.736572 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-9a58e9a1-4e91-4ff0-94bf-bb31eaa3e897 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b77623616a094df29ee7a94f74d7bfee', 'request_id': 'req-9a58e9a1-4e91-4ff0-94bf-bb31eaa3e897'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:04.743799 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-9a58e9a1-4e91-4ff0-94bf-bb31eaa3e897 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b77623616a094df29ee7a94f74d7bfee', 'request_id': 'req-9a58e9a1-4e91-4ff0-94bf-bb31eaa3e897'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:04.830923 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-9a58e9a1-4e91-4ff0-94bf-bb31eaa3e897 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b77623616a094df29ee7a94f74d7bfee', 'request_id': 'req-9a58e9a1-4e91-4ff0-94bf-bb31eaa3e897'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:04.843820 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-9a58e9a1-4e91-4ff0-94bf-bb31eaa3e897 admin admin] The token cache is being invalidate because project b77623616a094df29ee7a94f74d7bfee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 17 19:31:04.847784 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-9a58e9a1-4e91-4ff0-94bf-bb31eaa3e897 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:04.848646 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-9a58e9a1-4e91-4ff0-94bf-bb31eaa3e897 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:04.850680 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1093/2192] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 17 19:31:04 2022] DELETE /identity/v3/projects/b77623616a094df29ee7a94f74d7bfee => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:31:04.855302 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6610faaa-f92f-487a-98ac-ea41ca9449d1 service neutron] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:04.870900 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6610faaa-f92f-487a-98ac-ea41ca9449d1 service neutron] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:04.927542 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c677d47-29a6-4722-8c30-15b980554c94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:04.928409 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c677d47-29a6-4722-8c30-15b980554c94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:04.928682 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c677d47-29a6-4722-8c30-15b980554c94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:04.928946 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c677d47-29a6-4722-8c30-15b980554c94 admin admin] PATH_INFO: `/v3/users/3fa02ec5338442fcb560461b455e3346` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:04.930173 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c677d47-29a6-4722-8c30-15b980554c94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3fa02ec5338442fcb560461b455e3346)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:04.932156 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c677d47-29a6-4722-8c30-15b980554c94 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:05.131735 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-5c677d47-29a6-4722-8c30-15b980554c94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3fa02ec5338442fcb560461b455e3346'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:05.139064 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-5c677d47-29a6-4722-8c30-15b980554c94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3fa02ec5338442fcb560461b455e3346'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:05.150036 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a58e9a1-4e91-4ff0-94bf-bb31eaa3e897 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:05.151662 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-9a58e9a1-4e91-4ff0-94bf-bb31eaa3e897 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:05.151778 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-5c677d47-29a6-4722-8c30-15b980554c94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3fa02ec5338442fcb560461b455e3346'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:05.152485 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-5c677d47-29a6-4722-8c30-15b980554c94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3fa02ec5338442fcb560461b455e3346'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:05.153151 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-5c677d47-29a6-4722-8c30-15b980554c94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3fa02ec5338442fcb560461b455e3346'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:05.165326 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a58e9a1-4e91-4ff0-94bf-bb31eaa3e897 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:05.170525 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ac6c13-2e6e-412a-a57f-0d624fdce5a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:05.171411 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ac6c13-2e6e-412a-a57f-0d624fdce5a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:05.171781 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ac6c13-2e6e-412a-a57f-0d624fdce5a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:05.172005 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ac6c13-2e6e-412a-a57f-0d624fdce5a8 admin admin] PATH_INFO: `/v3/users/19becca57a494cc0bc8fc68bf9ad8c10` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:05.173191 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ac6c13-2e6e-412a-a57f-0d624fdce5a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=19becca57a494cc0bc8fc68bf9ad8c10)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:05.175123 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ac6c13-2e6e-412a-a57f-0d624fdce5a8 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:05.259999 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1100/2193] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 17 19:31:04 2022] DELETE /identity/v3/users/3fa02ec5338442fcb560461b455e3346 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 17 19:31:05.360302 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-26ac6c13-2e6e-412a-a57f-0d624fdce5a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19becca57a494cc0bc8fc68bf9ad8c10'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:05.360425 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-26ac6c13-2e6e-412a-a57f-0d624fdce5a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19becca57a494cc0bc8fc68bf9ad8c10'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:05.374265 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-26ac6c13-2e6e-412a-a57f-0d624fdce5a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19becca57a494cc0bc8fc68bf9ad8c10'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:05.459058 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-26ac6c13-2e6e-412a-a57f-0d624fdce5a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19becca57a494cc0bc8fc68bf9ad8c10'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:05.467199 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-26ac6c13-2e6e-412a-a57f-0d624fdce5a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19becca57a494cc0bc8fc68bf9ad8c10'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:05.474646 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1094/2194] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 17 19:31:05 2022] DELETE /identity/v3/users/19becca57a494cc0bc8fc68bf9ad8c10 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 17 19:31:05.479125 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa372a70-ec0c-4c67-87fa-c4665c2aa4a1 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:05.479635 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa372a70-ec0c-4c67-87fa-c4665c2aa4a1 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:05.481232 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa372a70-ec0c-4c67-87fa-c4665c2aa4a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:05.504114 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-aa372a70-ec0c-4c67-87fa-c4665c2aa4a1 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:31:05.505590 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-aa372a70-ec0c-4c67-87fa-c4665c2aa4a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:05.565788 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1101/2195] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:31:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:31:05.575990 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ac6c13-2e6e-412a-a57f-0d624fdce5a8 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:05.579691 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-26ac6c13-2e6e-412a-a57f-0d624fdce5a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:05.657075 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ac6c13-2e6e-412a-a57f-0d624fdce5a8 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:05.661364 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff41b7c-d1fe-41d2-b3c1-d959742fe64a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:05.662015 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff41b7c-d1fe-41d2-b3c1-d959742fe64a service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:05.662358 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff41b7c-d1fe-41d2-b3c1-d959742fe64a service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:05.662667 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff41b7c-d1fe-41d2-b3c1-d959742fe64a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:05.672619 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff41b7c-d1fe-41d2-b3c1-d959742fe64a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:05.674292 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff41b7c-d1fe-41d2-b3c1-d959742fe64a service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:05.684444 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1095/2196] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:31:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:31:05.869255 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa372a70-ec0c-4c67-87fa-c4665c2aa4a1 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:05.870801 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-aa372a70-ec0c-4c67-87fa-c4665c2aa4a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:05.882895 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa372a70-ec0c-4c67-87fa-c4665c2aa4a1 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:05.892679 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988e22b8-ffe2-4663-b6f3-5a625e301567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:05.897818 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988e22b8-ffe2-4663-b6f3-5a625e301567 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:05.897818 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988e22b8-ffe2-4663-b6f3-5a625e301567 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:05.897818 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988e22b8-ffe2-4663-b6f3-5a625e301567 admin admin] PATH_INFO: `/v3/projects/b1b76b871deb4435a4b3ee44b488e05e` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:05.897818 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988e22b8-ffe2-4663-b6f3-5a625e301567 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1b76b871deb4435a4b3ee44b488e05e)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:05.897818 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988e22b8-ffe2-4663-b6f3-5a625e301567 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:06.044687 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-988e22b8-ffe2-4663-b6f3-5a625e301567 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1b76b871deb4435a4b3ee44b488e05e', 'request_id': 'req-988e22b8-ffe2-4663-b6f3-5a625e301567'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:06.050980 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-988e22b8-ffe2-4663-b6f3-5a625e301567 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1b76b871deb4435a4b3ee44b488e05e', 'request_id': 'req-988e22b8-ffe2-4663-b6f3-5a625e301567'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:06.059084 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-988e22b8-ffe2-4663-b6f3-5a625e301567 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1b76b871deb4435a4b3ee44b488e05e', 'request_id': 'req-988e22b8-ffe2-4663-b6f3-5a625e301567'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:06.142506 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-988e22b8-ffe2-4663-b6f3-5a625e301567 admin admin] The token cache is being invalidate because project b1b76b871deb4435a4b3ee44b488e05e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 17 19:31:06.143217 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-988e22b8-ffe2-4663-b6f3-5a625e301567 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:06.144611 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-988e22b8-ffe2-4663-b6f3-5a625e301567 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:06.152747 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1102/2197] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 17 19:31:05 2022] DELETE /identity/v3/projects/b1b76b871deb4435a4b3ee44b488e05e => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:31:06.154563 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a532a560-89d0-4c33-a530-3f44e31eb1ee None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:06.155059 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a532a560-89d0-4c33-a530-3f44e31eb1ee None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:06.155158 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a532a560-89d0-4c33-a530-3f44e31eb1ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:06.179041 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-a532a560-89d0-4c33-a530-3f44e31eb1ee None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:31:06.179041 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a532a560-89d0-4c33-a530-3f44e31eb1ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:06.227759 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988e22b8-ffe2-4663-b6f3-5a625e301567 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:06.229376 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-988e22b8-ffe2-4663-b6f3-5a625e301567 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:06.236492 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1096/2198] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:31:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:31:06.243781 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988e22b8-ffe2-4663-b6f3-5a625e301567 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:06.244012 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ca31f5-e148-43b9-b658-702465961125 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:06.247534 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ca31f5-e148-43b9-b658-702465961125 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:06.247534 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ca31f5-e148-43b9-b658-702465961125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:06.248742 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87d5c6a-e3c7-4d85-ad03-e0b59dcf2161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:06.250093 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87d5c6a-e3c7-4d85-ad03-e0b59dcf2161 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:06.250377 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87d5c6a-e3c7-4d85-ad03-e0b59dcf2161 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:06.250650 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87d5c6a-e3c7-4d85-ad03-e0b59dcf2161 admin admin] PATH_INFO: `/v3/projects/f00cf662443f43ef951dd1aab8f35d7d` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:06.251984 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87d5c6a-e3c7-4d85-ad03-e0b59dcf2161 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f00cf662443f43ef951dd1aab8f35d7d)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:06.254487 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87d5c6a-e3c7-4d85-ad03-e0b59dcf2161 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:06.264597 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-13ca31f5-e148-43b9-b658-702465961125 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:31:06.265798 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-13ca31f5-e148-43b9-b658-702465961125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:06.274081 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1097/2199] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:31:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 17 19:31:06.278918 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ca31f5-e148-43b9-b658-702465961125 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:06.288282 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ca31f5-e148-43b9-b658-702465961125 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:06.293746 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719606f1-8655-499e-ab44-912717ad49ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:06.294431 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719606f1-8655-499e-ab44-912717ad49ca admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:06.294712 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719606f1-8655-499e-ab44-912717ad49ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:06.294984 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719606f1-8655-499e-ab44-912717ad49ca admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:06.295749 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719606f1-8655-499e-ab44-912717ad49ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:31:06.296056 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719606f1-8655-499e-ab44-912717ad49ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:06.297746 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719606f1-8655-499e-ab44-912717ad49ca admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:06.312603 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1098/2200] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 17 19:31:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:31:06.316656 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719606f1-8655-499e-ab44-912717ad49ca admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:06.327393 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719606f1-8655-499e-ab44-912717ad49ca admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:06.332213 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440a1264-32a3-41fd-9da9-7f2fd82c10b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:06.332852 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440a1264-32a3-41fd-9da9-7f2fd82c10b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:06.333127 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440a1264-32a3-41fd-9da9-7f2fd82c10b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:06.333422 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440a1264-32a3-41fd-9da9-7f2fd82c10b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:06.334122 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440a1264-32a3-41fd-9da9-7f2fd82c10b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:06.336184 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440a1264-32a3-41fd-9da9-7f2fd82c10b2 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:06.393322 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-c87d5c6a-e3c7-4d85-ad03-e0b59dcf2161 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f00cf662443f43ef951dd1aab8f35d7d', 'request_id': 'req-c87d5c6a-e3c7-4d85-ad03-e0b59dcf2161'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:06.399548 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-c87d5c6a-e3c7-4d85-ad03-e0b59dcf2161 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f00cf662443f43ef951dd1aab8f35d7d', 'request_id': 'req-c87d5c6a-e3c7-4d85-ad03-e0b59dcf2161'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:06.405585 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-c87d5c6a-e3c7-4d85-ad03-e0b59dcf2161 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f00cf662443f43ef951dd1aab8f35d7d', 'request_id': 'req-c87d5c6a-e3c7-4d85-ad03-e0b59dcf2161'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:06.460818 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1099/2201] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 17 19:31:06 2022] POST /identity/v3/projects => generated 365 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 17 19:31:06.465660 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440a1264-32a3-41fd-9da9-7f2fd82c10b2 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:06.486065 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-c87d5c6a-e3c7-4d85-ad03-e0b59dcf2161 admin admin] The token cache is being invalidate because project f00cf662443f43ef951dd1aab8f35d7d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 17 19:31:06.486341 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-c87d5c6a-e3c7-4d85-ad03-e0b59dcf2161 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:06.487172 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-c87d5c6a-e3c7-4d85-ad03-e0b59dcf2161 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:06.489237 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1103/2202] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 17 19:31:06 2022] DELETE /identity/v3/projects/f00cf662443f43ef951dd1aab8f35d7d => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 17 19:31:06.494580 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eff2196-e6d0-474b-b4c7-e1690f1300a9 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:06.495146 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eff2196-e6d0-474b-b4c7-e1690f1300a9 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:06.495607 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eff2196-e6d0-474b-b4c7-e1690f1300a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:06.523812 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-3eff2196-e6d0-474b-b4c7-e1690f1300a9 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:31:06.525665 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-3eff2196-e6d0-474b-b4c7-e1690f1300a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:06.561639 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440a1264-32a3-41fd-9da9-7f2fd82c10b2 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:06.580067 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3156298-0dcd-44b5-bffb-6a9804637610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:06.583323 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3156298-0dcd-44b5-bffb-6a9804637610 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:06.583323 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3156298-0dcd-44b5-bffb-6a9804637610 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:06.583323 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3156298-0dcd-44b5-bffb-6a9804637610 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:06.584158 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3156298-0dcd-44b5-bffb-6a9804637610 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:06.586307 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3156298-0dcd-44b5-bffb-6a9804637610 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:06.622380 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1104/2203] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:31:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 17 19:31:06.630067 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9613933-97ea-4ae2-8018-a6b8182f8d03 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:06.630474 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9613933-97ea-4ae2-8018-a6b8182f8d03 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:06.630752 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9613933-97ea-4ae2-8018-a6b8182f8d03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:06.649755 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-d9613933-97ea-4ae2-8018-a6b8182f8d03 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:31:06.651011 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-d9613933-97ea-4ae2-8018-a6b8182f8d03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:06.662590 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1105/2204] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:31:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:31:06.663811 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9613933-97ea-4ae2-8018-a6b8182f8d03 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:06.674064 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9613933-97ea-4ae2-8018-a6b8182f8d03 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:06.675588 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1100/2205] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 17 19:31:06 2022] POST /identity/v3/users => generated 283 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:31:06.679008 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3a0eef-d905-4d95-882c-8932c07a21c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:06.679695 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3a0eef-d905-4d95-882c-8932c07a21c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:06.679811 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3156298-0dcd-44b5-bffb-6a9804637610 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:06.679973 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3a0eef-d905-4d95-882c-8932c07a21c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:06.680245 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3a0eef-d905-4d95-882c-8932c07a21c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:06.680973 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3a0eef-d905-4d95-882c-8932c07a21c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:31:06.681277 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3a0eef-d905-4d95-882c-8932c07a21c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:06.681383 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b3156298-0dcd-44b5-bffb-6a9804637610 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:06.682923 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3a0eef-d905-4d95-882c-8932c07a21c2 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:06.694457 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3156298-0dcd-44b5-bffb-6a9804637610 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:06.699159 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1106/2206] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 17 19:31:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:31:06.707731 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9710dcf-e672-4cd6-9c10-5e13a1e34104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:06.707731 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9710dcf-e672-4cd6-9c10-5e13a1e34104 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:06.707731 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9710dcf-e672-4cd6-9c10-5e13a1e34104 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:06.707731 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9710dcf-e672-4cd6-9c10-5e13a1e34104 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:06.708670 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9710dcf-e672-4cd6-9c10-5e13a1e34104 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:06.708670 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9710dcf-e672-4cd6-9c10-5e13a1e34104 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:06.712149 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3a0eef-d905-4d95-882c-8932c07a21c2 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:06.723223 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1101/2207] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:31:06 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 17 19:31:06.724864 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9710dcf-e672-4cd6-9c10-5e13a1e34104 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:06.725293 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3a0eef-d905-4d95-882c-8932c07a21c2 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:06.730307 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186f7e17-272c-4211-a21e-9f0f2341e3b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:06.730987 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186f7e17-272c-4211-a21e-9f0f2341e3b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:06.731271 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186f7e17-272c-4211-a21e-9f0f2341e3b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:06.731576 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186f7e17-272c-4211-a21e-9f0f2341e3b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:06.732289 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186f7e17-272c-4211-a21e-9f0f2341e3b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:06.734217 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186f7e17-272c-4211-a21e-9f0f2341e3b3 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:06.744109 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9710dcf-e672-4cd6-9c10-5e13a1e34104 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:06.749720 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bf7e21-b87d-4848-87a4-3e5c1c74ea0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:06.750396 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bf7e21-b87d-4848-87a4-3e5c1c74ea0f admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:06.750674 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bf7e21-b87d-4848-87a4-3e5c1c74ea0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:06.751115 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bf7e21-b87d-4848-87a4-3e5c1c74ea0f admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:06.751960 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bf7e21-b87d-4848-87a4-3e5c1c74ea0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:06.753387 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bf7e21-b87d-4848-87a4-3e5c1c74ea0f admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:06.764837 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1102/2208] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:31:06 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 17 19:31:06.769341 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bf7e21-b87d-4848-87a4-3e5c1c74ea0f admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:06.785626 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bf7e21-b87d-4848-87a4-3e5c1c74ea0f admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:06.790917 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9c1715-0564-40cb-bbcb-0d527205ec63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:06.792095 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9c1715-0564-40cb-bbcb-0d527205ec63 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:06.792444 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9c1715-0564-40cb-bbcb-0d527205ec63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:06.792778 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9c1715-0564-40cb-bbcb-0d527205ec63 admin admin] PATH_INFO: `/v3/projects/def5a0bf5f2843feafabda0ba4b00b72/users/8d88717e79ef4ce8802e9c41ac20142b/roles/114e6a16e63447d1924707ace5aace25` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:06.809921 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9c1715-0564-40cb-bbcb-0d527205ec63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=def5a0bf5f2843feafabda0ba4b00b72, user_id=8d88717e79ef4ce8802e9c41ac20142b, role_id=114e6a16e63447d1924707ace5aace25)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:06.821920 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9c1715-0564-40cb-bbcb-0d527205ec63 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:06.867394 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1107/2209] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 17 19:31:06 2022] POST /identity/v3/projects => generated 365 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 17 19:31:06.873128 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186f7e17-272c-4211-a21e-9f0f2341e3b3 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:06.935395 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186f7e17-272c-4211-a21e-9f0f2341e3b3 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:06.943761 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1103/2210] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:31:06 2022] PUT /identity/v3/projects/def5a0bf5f2843feafabda0ba4b00b72/users/8d88717e79ef4ce8802e9c41ac20142b/roles/114e6a16e63447d1924707ace5aace25 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 17 19:31:06.951720 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b7947c-fcfe-4d99-83db-a20917bb4b58 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:06.954936 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b7947c-fcfe-4d99-83db-a20917bb4b58 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:06.954936 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b7947c-fcfe-4d99-83db-a20917bb4b58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:06.984461 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-c2b7947c-fcfe-4d99-83db-a20917bb4b58 None None] MFA Rules not processed for user `8d88717e79ef4ce8802e9c41ac20142b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:31:06.985841 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c2b7947c-fcfe-4d99-83db-a20917bb4b58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:06.997533 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573abf50-2e1a-4373-9f71-31304dd85996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:06.998198 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573abf50-2e1a-4373-9f71-31304dd85996 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:06.998480 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573abf50-2e1a-4373-9f71-31304dd85996 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:06.998764 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573abf50-2e1a-4373-9f71-31304dd85996 admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:06.999448 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573abf50-2e1a-4373-9f71-31304dd85996 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:07.001588 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573abf50-2e1a-4373-9f71-31304dd85996 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:07.065162 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1104/2211] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:31:06 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 17 19:31:07.069678 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b7947c-fcfe-4d99-83db-a20917bb4b58 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:07.081317 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b7947c-fcfe-4d99-83db-a20917bb4b58 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:07.087212 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5f81c8-a80d-40da-90f8-4f7b47304e47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:07.087212 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5f81c8-a80d-40da-90f8-4f7b47304e47 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:07.087212 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5f81c8-a80d-40da-90f8-4f7b47304e47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:07.087212 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5f81c8-a80d-40da-90f8-4f7b47304e47 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:07.088022 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5f81c8-a80d-40da-90f8-4f7b47304e47 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:07.089973 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5f81c8-a80d-40da-90f8-4f7b47304e47 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:07.112044 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1108/2212] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 17 19:31:06 2022] POST /identity/v3/users => generated 283 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:31:07.121038 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573abf50-2e1a-4373-9f71-31304dd85996 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:07.130571 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573abf50-2e1a-4373-9f71-31304dd85996 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:07.135112 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e608138-fad1-426d-97d1-ac9748eb1d8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:07.135831 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e608138-fad1-426d-97d1-ac9748eb1d8d admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:07.136105 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e608138-fad1-426d-97d1-ac9748eb1d8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:07.136375 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e608138-fad1-426d-97d1-ac9748eb1d8d admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:07.137081 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e608138-fad1-426d-97d1-ac9748eb1d8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:07.138651 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e608138-fad1-426d-97d1-ac9748eb1d8d admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:07.148350 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1109/2213] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:31:07 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 17 19:31:07.152302 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e608138-fad1-426d-97d1-ac9748eb1d8d admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:07.163006 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e608138-fad1-426d-97d1-ac9748eb1d8d admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:07.172714 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167085c1-6552-4de2-a5bb-fac2b6a6e8ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:07.173137 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1105/2214] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 17 19:31:07 2022] POST /identity/v3/projects => generated 365 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 17 19:31:07.173613 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167085c1-6552-4de2-a5bb-fac2b6a6e8ad admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:07.173707 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167085c1-6552-4de2-a5bb-fac2b6a6e8ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:07.173981 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167085c1-6552-4de2-a5bb-fac2b6a6e8ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:07.174694 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167085c1-6552-4de2-a5bb-fac2b6a6e8ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:07.176721 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167085c1-6552-4de2-a5bb-fac2b6a6e8ad admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:07.177349 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5f81c8-a80d-40da-90f8-4f7b47304e47 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:07.188302 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1110/2215] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:31:07 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 17 19:31:07.192260 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167085c1-6552-4de2-a5bb-fac2b6a6e8ad admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:07.203241 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167085c1-6552-4de2-a5bb-fac2b6a6e8ad admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:07.248839 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5f81c8-a80d-40da-90f8-4f7b47304e47 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:07.256176 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25305f51-bf20-4edd-b9ed-d9966c6bcef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:07.258048 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25305f51-bf20-4edd-b9ed-d9966c6bcef4 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:07.258327 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25305f51-bf20-4edd-b9ed-d9966c6bcef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:07.258650 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25305f51-bf20-4edd-b9ed-d9966c6bcef4 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:07.259351 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25305f51-bf20-4edd-b9ed-d9966c6bcef4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:07.261670 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25305f51-bf20-4edd-b9ed-d9966c6bcef4 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:07.286929 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94be510d-0b4b-47f7-9696-7e7a8db1eaf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:07.287916 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94be510d-0b4b-47f7-9696-7e7a8db1eaf0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:07.288360 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94be510d-0b4b-47f7-9696-7e7a8db1eaf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:07.289178 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94be510d-0b4b-47f7-9696-7e7a8db1eaf0 admin admin] PATH_INFO: `/v3/projects/83c30e8ab6e2481ca4dd41bc68d93846/users/910494d191cc4e939c67dd546a914c08/roles/114e6a16e63447d1924707ace5aace25` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:07.304995 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94be510d-0b4b-47f7-9696-7e7a8db1eaf0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83c30e8ab6e2481ca4dd41bc68d93846, user_id=910494d191cc4e939c67dd546a914c08, role_id=114e6a16e63447d1924707ace5aace25)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:07.307754 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94be510d-0b4b-47f7-9696-7e7a8db1eaf0 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:07.369704 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1106/2216] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 17 19:31:07 2022] POST /identity/v3/users => generated 289 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 17 19:31:07.373399 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25305f51-bf20-4edd-b9ed-d9966c6bcef4 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:07.383143 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25305f51-bf20-4edd-b9ed-d9966c6bcef4 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:07.388371 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3867088-5717-4243-a688-b7651273406b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:07.389016 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3867088-5717-4243-a688-b7651273406b admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:07.389300 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3867088-5717-4243-a688-b7651273406b admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:07.389734 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3867088-5717-4243-a688-b7651273406b admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:07.390419 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3867088-5717-4243-a688-b7651273406b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:07.392055 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3867088-5717-4243-a688-b7651273406b admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:07.397579 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1111/2217] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:31:07 2022] PUT /identity/v3/projects/83c30e8ab6e2481ca4dd41bc68d93846/users/910494d191cc4e939c67dd546a914c08/roles/114e6a16e63447d1924707ace5aace25 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 17 19:31:07.404443 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1107/2218] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:31:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:31:07.413919 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62232b8a-2650-417b-af91-2d12006f78c2 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:07.414405 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62232b8a-2650-417b-af91-2d12006f78c2 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:07.414759 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62232b8a-2650-417b-af91-2d12006f78c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:07.416561 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3867088-5717-4243-a688-b7651273406b admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:07.427035 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3867088-5717-4243-a688-b7651273406b admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:07.446221 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-62232b8a-2650-417b-af91-2d12006f78c2 None None] MFA Rules not processed for user `910494d191cc4e939c67dd546a914c08`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:31:07.447848 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-62232b8a-2650-417b-af91-2d12006f78c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:07.486110 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c3fee7-96b6-47f8-94e9-0f7d7aba9607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:07.487050 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c3fee7-96b6-47f8-94e9-0f7d7aba9607 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:07.492142 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c3fee7-96b6-47f8-94e9-0f7d7aba9607 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:07.492142 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c3fee7-96b6-47f8-94e9-0f7d7aba9607 admin admin] PATH_INFO: `/v3/projects/8d28e5936a9a42cdb5f921e4a6c947f8/users/d129b5884ac74d9f90a74dcdfc82678f/roles/60af5088793e4379b88821a86552a344` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:07.504755 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c3fee7-96b6-47f8-94e9-0f7d7aba9607 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d28e5936a9a42cdb5f921e4a6c947f8, user_id=d129b5884ac74d9f90a74dcdfc82678f, role_id=60af5088793e4379b88821a86552a344)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:07.509283 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c3fee7-96b6-47f8-94e9-0f7d7aba9607 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:07.531609 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1112/2219] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:31:07 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 17 19:31:07.545250 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62232b8a-2650-417b-af91-2d12006f78c2 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:07.551804 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-62232b8a-2650-417b-af91-2d12006f78c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:07.598289 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1108/2220] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:31:07 2022] PUT /identity/v3/projects/8d28e5936a9a42cdb5f921e4a6c947f8/users/d129b5884ac74d9f90a74dcdfc82678f/roles/60af5088793e4379b88821a86552a344 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:31:07.602687 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c3fee7-96b6-47f8-94e9-0f7d7aba9607 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:07.616104 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62232b8a-2650-417b-af91-2d12006f78c2 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:07.675303 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404aa27b-8ff8-4460-b9cf-a6a29f11a502 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd05409b38ae346458abfe41ebda1c59e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:07.676324 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404aa27b-8ff8-4460-b9cf-a6a29f11a502 service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:07.676760 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404aa27b-8ff8-4460-b9cf-a6a29f11a502 service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:07.678622 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404aa27b-8ff8-4460-b9cf-a6a29f11a502 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:07.689804 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c3fee7-96b6-47f8-94e9-0f7d7aba9607 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:07.696417 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeda899-a741-4e95-8822-a2e367549cad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:07.697150 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeda899-a741-4e95-8822-a2e367549cad admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:07.697498 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeda899-a741-4e95-8822-a2e367549cad admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:07.698060 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeda899-a741-4e95-8822-a2e367549cad admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:07.699931 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eeda899-a741-4e95-8822-a2e367549cad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:07.701599 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eeda899-a741-4e95-8822-a2e367549cad admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:07.715194 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1109/2221] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:31:07 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 17 19:31:07.718843 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeda899-a741-4e95-8822-a2e367549cad admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:07.732579 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeda899-a741-4e95-8822-a2e367549cad admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:07.738164 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71e9cd1-a127-4d74-8955-b4a573880fb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:07.739063 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71e9cd1-a127-4d74-8955-b4a573880fb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:07.739342 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71e9cd1-a127-4d74-8955-b4a573880fb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:07.739656 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71e9cd1-a127-4d74-8955-b4a573880fb6 admin admin] PATH_INFO: `/v3/projects/8d28e5936a9a42cdb5f921e4a6c947f8/users/d129b5884ac74d9f90a74dcdfc82678f/roles/60af5088793e4379b88821a86552a344` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:07.741962 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71e9cd1-a127-4d74-8955-b4a573880fb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d28e5936a9a42cdb5f921e4a6c947f8, user_id=d129b5884ac74d9f90a74dcdfc82678f, role_id=60af5088793e4379b88821a86552a344)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:07.744743 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71e9cd1-a127-4d74-8955-b4a573880fb6 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:07.763464 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1110/2222] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:31:07 2022] PUT /identity/v3/projects/8d28e5936a9a42cdb5f921e4a6c947f8/users/d129b5884ac74d9f90a74dcdfc82678f/roles/60af5088793e4379b88821a86552a344 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 17 19:31:07.764049 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404aa27b-8ff8-4460-b9cf-a6a29f11a502 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:07.765607 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404aa27b-8ff8-4460-b9cf-a6a29f11a502 service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:07.777440 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1113/2223] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:31:07 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 17 19:31:07.783684 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4d58a9-612a-42c2-8af0-20b69eea2439 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:07.784280 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4d58a9-612a-42c2-8af0-20b69eea2439 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:07.784280 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4d58a9-612a-42c2-8af0-20b69eea2439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:07.818397 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-3d4d58a9-612a-42c2-8af0-20b69eea2439 None None] MFA Rules not processed for user `d129b5884ac74d9f90a74dcdfc82678f`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:31:07.820480 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-3d4d58a9-612a-42c2-8af0-20b69eea2439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:07.917709 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1111/2224] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:31:07 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 17 19:31:08.017173 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404aa27b-8ff8-4460-b9cf-a6a29f11a502 service cinder] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:08.018570 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-404aa27b-8ff8-4460-b9cf-a6a29f11a502 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:08.029304 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404aa27b-8ff8-4460-b9cf-a6a29f11a502 service cinder] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:08.077406 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9cc384-660f-4f72-84b1-12b6753315e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd05409b38ae346458abfe41ebda1c59e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:08.078036 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9cc384-660f-4f72-84b1-12b6753315e3 service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:08.078314 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9cc384-660f-4f72-84b1-12b6753315e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:08.078580 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9cc384-660f-4f72-84b1-12b6753315e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:08.087405 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9cc384-660f-4f72-84b1-12b6753315e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:08.088944 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9cc384-660f-4f72-84b1-12b6753315e3 service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:08.099543 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1114/2225] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:31:08 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:31:08.447622 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4d58a9-612a-42c2-8af0-20b69eea2439 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:08.461914 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4d58a9-612a-42c2-8af0-20b69eea2439 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:08.465592 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cb355b-33b2-44d0-b5ce-dfba2035c87a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd05409b38ae346458abfe41ebda1c59e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:08.466294 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cb355b-33b2-44d0-b5ce-dfba2035c87a service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:08.466679 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cb355b-33b2-44d0-b5ce-dfba2035c87a service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:08.467018 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cb355b-33b2-44d0-b5ce-dfba2035c87a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:08.540968 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cb355b-33b2-44d0-b5ce-dfba2035c87a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:08.543024 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cb355b-33b2-44d0-b5ce-dfba2035c87a service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:08.554012 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1112/2226] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:31:08 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:31:10.116939 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9cc384-660f-4f72-84b1-12b6753315e3 service cinder] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:10.118490 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-ea9cc384-660f-4f72-84b1-12b6753315e3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:10.236459 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9cc384-660f-4f72-84b1-12b6753315e3 service cinder] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:10.242413 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91de6a2-1fc0-4e1a-bf44-f9e7324fcaea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:10.243601 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91de6a2-1fc0-4e1a-bf44-f9e7324fcaea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:10.244031 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91de6a2-1fc0-4e1a-bf44-f9e7324fcaea admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:10.244314 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91de6a2-1fc0-4e1a-bf44-f9e7324fcaea admin admin] PATH_INFO: `/v3/users/e0d5404a4851455d9b82a9f21a641625` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:10.247273 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91de6a2-1fc0-4e1a-bf44-f9e7324fcaea admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0d5404a4851455d9b82a9f21a641625)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:10.250231 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91de6a2-1fc0-4e1a-bf44-f9e7324fcaea admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:10.453592 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-d91de6a2-1fc0-4e1a-bf44-f9e7324fcaea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0d5404a4851455d9b82a9f21a641625'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:10.460912 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-d91de6a2-1fc0-4e1a-bf44-f9e7324fcaea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0d5404a4851455d9b82a9f21a641625'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:10.473870 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-d91de6a2-1fc0-4e1a-bf44-f9e7324fcaea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0d5404a4851455d9b82a9f21a641625'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:10.474756 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-d91de6a2-1fc0-4e1a-bf44-f9e7324fcaea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0d5404a4851455d9b82a9f21a641625'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:10.475772 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-d91de6a2-1fc0-4e1a-bf44-f9e7324fcaea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0d5404a4851455d9b82a9f21a641625'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:10.539853 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1115/2227] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 17 19:31:10 2022] DELETE /identity/v3/users/e0d5404a4851455d9b82a9f21a641625 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:31:10.792866 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cb355b-33b2-44d0-b5ce-dfba2035c87a service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:10.803600 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cb355b-33b2-44d0-b5ce-dfba2035c87a service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:10.862052 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ac7a9a-c8ee-4e6f-862c-55239e6963f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:10.862945 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac7a9a-c8ee-4e6f-862c-55239e6963f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:10.863232 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac7a9a-c8ee-4e6f-862c-55239e6963f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:10.863516 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac7a9a-c8ee-4e6f-862c-55239e6963f7 admin admin] PATH_INFO: `/v3/projects/f18adcec58b14844bec04657953b5d10` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:10.864758 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ac7a9a-c8ee-4e6f-862c-55239e6963f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f18adcec58b14844bec04657953b5d10)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:10.866753 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ac7a9a-c8ee-4e6f-862c-55239e6963f7 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:11.286971 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-a8ac7a9a-c8ee-4e6f-862c-55239e6963f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f18adcec58b14844bec04657953b5d10', 'request_id': 'req-a8ac7a9a-c8ee-4e6f-862c-55239e6963f7'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:11.290523 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-a8ac7a9a-c8ee-4e6f-862c-55239e6963f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f18adcec58b14844bec04657953b5d10', 'request_id': 'req-a8ac7a9a-c8ee-4e6f-862c-55239e6963f7'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:11.496414 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-a8ac7a9a-c8ee-4e6f-862c-55239e6963f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f18adcec58b14844bec04657953b5d10', 'request_id': 'req-a8ac7a9a-c8ee-4e6f-862c-55239e6963f7'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:11.514415 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-a8ac7a9a-c8ee-4e6f-862c-55239e6963f7 admin admin] The token cache is being invalidate because project f18adcec58b14844bec04657953b5d10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 17 19:31:11.514795 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-a8ac7a9a-c8ee-4e6f-862c-55239e6963f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:11.515528 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-a8ac7a9a-c8ee-4e6f-862c-55239e6963f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:11.517223 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1113/2228] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 17 19:31:10 2022] DELETE /identity/v3/projects/f18adcec58b14844bec04657953b5d10 => generated 0 bytes in 725 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:31:11.521376 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91de6a2-1fc0-4e1a-bf44-f9e7324fcaea admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:11.523024 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-d91de6a2-1fc0-4e1a-bf44-f9e7324fcaea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:11.537479 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91de6a2-1fc0-4e1a-bf44-f9e7324fcaea admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:11.594784 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6941adbf-4333-43d6-a85e-c3be8e3d78d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:11.595842 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6941adbf-4333-43d6-a85e-c3be8e3d78d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:11.596144 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6941adbf-4333-43d6-a85e-c3be8e3d78d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:11.596457 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6941adbf-4333-43d6-a85e-c3be8e3d78d6 admin admin] PATH_INFO: `/v3/users/b1d22d948b1e4e10beafaeb652ce4ad9` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:11.598079 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6941adbf-4333-43d6-a85e-c3be8e3d78d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1d22d948b1e4e10beafaeb652ce4ad9)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:11.600324 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6941adbf-4333-43d6-a85e-c3be8e3d78d6 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:12.035558 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-6941adbf-4333-43d6-a85e-c3be8e3d78d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1d22d948b1e4e10beafaeb652ce4ad9'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:12.042734 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-6941adbf-4333-43d6-a85e-c3be8e3d78d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1d22d948b1e4e10beafaeb652ce4ad9'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:12.054360 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-6941adbf-4333-43d6-a85e-c3be8e3d78d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1d22d948b1e4e10beafaeb652ce4ad9'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:12.055146 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-6941adbf-4333-43d6-a85e-c3be8e3d78d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1d22d948b1e4e10beafaeb652ce4ad9'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:12.055773 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-6941adbf-4333-43d6-a85e-c3be8e3d78d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1d22d948b1e4e10beafaeb652ce4ad9'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:12.220383 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1116/2229] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 17 19:31:11 2022] DELETE /identity/v3/users/b1d22d948b1e4e10beafaeb652ce4ad9 => generated 0 bytes in 699 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 17 19:31:12.498157 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ac7a9a-c8ee-4e6f-862c-55239e6963f7 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:12.502599 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a8ac7a9a-c8ee-4e6f-862c-55239e6963f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:12.529594 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ac7a9a-c8ee-4e6f-862c-55239e6963f7 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:12.587964 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12104f76-8651-4674-8a70-e27070f254c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:12.589577 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12104f76-8651-4674-8a70-e27070f254c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:12.589749 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12104f76-8651-4674-8a70-e27070f254c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:12.589986 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12104f76-8651-4674-8a70-e27070f254c6 admin admin] PATH_INFO: `/v3/projects/21af0c27992c45b7be22b686674afef9` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:12.591184 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12104f76-8651-4674-8a70-e27070f254c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=21af0c27992c45b7be22b686674afef9)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:12.593793 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12104f76-8651-4674-8a70-e27070f254c6 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:12.723206 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-12104f76-8651-4674-8a70-e27070f254c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21af0c27992c45b7be22b686674afef9', 'request_id': 'req-12104f76-8651-4674-8a70-e27070f254c6'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:12.728599 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-12104f76-8651-4674-8a70-e27070f254c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21af0c27992c45b7be22b686674afef9', 'request_id': 'req-12104f76-8651-4674-8a70-e27070f254c6'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:12.850251 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-12104f76-8651-4674-8a70-e27070f254c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21af0c27992c45b7be22b686674afef9', 'request_id': 'req-12104f76-8651-4674-8a70-e27070f254c6'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:12.863967 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-12104f76-8651-4674-8a70-e27070f254c6 admin admin] The token cache is being invalidate because project 21af0c27992c45b7be22b686674afef9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 17 19:31:12.864252 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-12104f76-8651-4674-8a70-e27070f254c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:12.865569 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-12104f76-8651-4674-8a70-e27070f254c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:12.867578 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1114/2230] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 17 19:31:12 2022] DELETE /identity/v3/projects/21af0c27992c45b7be22b686674afef9 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:31:12.872893 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6f5a37-d4ef-4fcc-8e90-e03acd4888cf None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:12.873364 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6f5a37-d4ef-4fcc-8e90-e03acd4888cf None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:12.873539 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6f5a37-d4ef-4fcc-8e90-e03acd4888cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:12.891819 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-ee6f5a37-d4ef-4fcc-8e90-e03acd4888cf None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:31:12.893074 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-ee6f5a37-d4ef-4fcc-8e90-e03acd4888cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:12.952186 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1117/2231] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:31:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 17 19:31:12.961931 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7cb18f-5cf6-4436-a08c-b71caf7dbac4 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:12.962213 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7cb18f-5cf6-4436-a08c-b71caf7dbac4 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:12.962492 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7cb18f-5cf6-4436-a08c-b71caf7dbac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:12.985963 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-8e7cb18f-5cf6-4436-a08c-b71caf7dbac4 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:31:12.987166 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-8e7cb18f-5cf6-4436-a08c-b71caf7dbac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:12.995862 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1115/2232] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:31:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:31:13.000295 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6f5a37-d4ef-4fcc-8e90-e03acd4888cf None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:13.009967 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6f5a37-d4ef-4fcc-8e90-e03acd4888cf None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:13.015137 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc4ef1d-5080-4528-954d-098df5487a0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:13.015827 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc4ef1d-5080-4528-954d-098df5487a0f admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:13.016111 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc4ef1d-5080-4528-954d-098df5487a0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:13.016387 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc4ef1d-5080-4528-954d-098df5487a0f admin admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:13.017228 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc4ef1d-5080-4528-954d-098df5487a0f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:31:13.017542 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc4ef1d-5080-4528-954d-098df5487a0f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:13.019131 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc4ef1d-5080-4528-954d-098df5487a0f admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:13.031176 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1118/2233] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 17 19:31:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:31:13.034931 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7cb18f-5cf6-4436-a08c-b71caf7dbac4 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:13.044986 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7cb18f-5cf6-4436-a08c-b71caf7dbac4 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:13.049928 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de3f715-087c-4be0-ba45-826ecdcf29b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:13.050599 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de3f715-087c-4be0-ba45-826ecdcf29b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:13.050878 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de3f715-087c-4be0-ba45-826ecdcf29b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:13.051163 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de3f715-087c-4be0-ba45-826ecdcf29b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:13.051873 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de3f715-087c-4be0-ba45-826ecdcf29b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:13.053915 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de3f715-087c-4be0-ba45-826ecdcf29b9 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:13.221250 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1116/2234] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 17 19:31:13 2022] POST /identity/v3/projects => generated 385 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:31:13.222830 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc4ef1d-5080-4528-954d-098df5487a0f admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:13.235008 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc4ef1d-5080-4528-954d-098df5487a0f admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:13.293371 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771e18d8-d170-40a7-9278-b7b64b7b0dd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:13.294034 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771e18d8-d170-40a7-9278-b7b64b7b0dd6 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:13.294312 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771e18d8-d170-40a7-9278-b7b64b7b0dd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:13.294579 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771e18d8-d170-40a7-9278-b7b64b7b0dd6 admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:13.295249 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771e18d8-d170-40a7-9278-b7b64b7b0dd6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:13.297321 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771e18d8-d170-40a7-9278-b7b64b7b0dd6 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:13.423018 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1119/2235] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 17 19:31:13 2022] POST /identity/v3/users => generated 293 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:31:13.427032 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de3f715-087c-4be0-ba45-826ecdcf29b9 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:13.436873 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de3f715-087c-4be0-ba45-826ecdcf29b9 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:13.441377 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6562ff-a51b-4d49-b8ba-2e0acb27697e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:13.442023 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6562ff-a51b-4d49-b8ba-2e0acb27697e admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:13.442303 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6562ff-a51b-4d49-b8ba-2e0acb27697e admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:13.442579 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6562ff-a51b-4d49-b8ba-2e0acb27697e admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:13.443299 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6562ff-a51b-4d49-b8ba-2e0acb27697e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:13.444973 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6562ff-a51b-4d49-b8ba-2e0acb27697e admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:13.455525 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1117/2236] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:31:13 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:31:13.459663 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771e18d8-d170-40a7-9278-b7b64b7b0dd6 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:13.470840 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771e18d8-d170-40a7-9278-b7b64b7b0dd6 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:13.475713 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4bdf6d-46cd-42fc-a9fb-e5d29c08d992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:13.476361 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4bdf6d-46cd-42fc-a9fb-e5d29c08d992 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:13.476701 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4bdf6d-46cd-42fc-a9fb-e5d29c08d992 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:13.476996 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4bdf6d-46cd-42fc-a9fb-e5d29c08d992 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:13.478406 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4bdf6d-46cd-42fc-a9fb-e5d29c08d992 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:13.480665 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4bdf6d-46cd-42fc-a9fb-e5d29c08d992 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:13.492761 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1120/2237] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:31:13 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:31:13.497311 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6562ff-a51b-4d49-b8ba-2e0acb27697e admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:13.507273 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6562ff-a51b-4d49-b8ba-2e0acb27697e admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:13.512811 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9ed11d-4f0e-408b-a86d-42b5fe16acf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:13.513671 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9ed11d-4f0e-408b-a86d-42b5fe16acf5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:13.513945 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9ed11d-4f0e-408b-a86d-42b5fe16acf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:13.514211 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9ed11d-4f0e-408b-a86d-42b5fe16acf5 admin admin] PATH_INFO: `/v3/projects/66b47f4795be411daf349cf57f8e3a15/users/b9a877937193493992f96c0bb1f92191/roles/114e6a16e63447d1924707ace5aace25` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:13.528718 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9ed11d-4f0e-408b-a86d-42b5fe16acf5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66b47f4795be411daf349cf57f8e3a15, user_id=b9a877937193493992f96c0bb1f92191, role_id=114e6a16e63447d1924707ace5aace25)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:13.531363 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9ed11d-4f0e-408b-a86d-42b5fe16acf5 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:13.613900 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1118/2238] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:31:13 2022] PUT /identity/v3/projects/66b47f4795be411daf349cf57f8e3a15/users/b9a877937193493992f96c0bb1f92191/roles/114e6a16e63447d1924707ace5aace25 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:31:13.624280 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f8cdd1-0a2a-4b39-bad1-d9847c5f8d55 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:13.624472 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f8cdd1-0a2a-4b39-bad1-d9847c5f8d55 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:13.624715 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f8cdd1-0a2a-4b39-bad1-d9847c5f8d55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:13.652903 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-47f8cdd1-0a2a-4b39-bad1-d9847c5f8d55 None None] MFA Rules not processed for user `b9a877937193493992f96c0bb1f92191`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:31:13.654170 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-47f8cdd1-0a2a-4b39-bad1-d9847c5f8d55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:13.748649 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1121/2239] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:31:13 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:31:13.919776 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9ed11d-4f0e-408b-a86d-42b5fe16acf5 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:13.921290 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-4f9ed11d-4f0e-408b-a86d-42b5fe16acf5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:13.977726 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9ed11d-4f0e-408b-a86d-42b5fe16acf5 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:13.981455 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c9efa6-90cd-4702-95d9-3161d125571f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd05409b38ae346458abfe41ebda1c59e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:13.982087 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c9efa6-90cd-4702-95d9-3161d125571f service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:13.982363 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c9efa6-90cd-4702-95d9-3161d125571f service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:13.982634 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c9efa6-90cd-4702-95d9-3161d125571f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:13.990593 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c9efa6-90cd-4702-95d9-3161d125571f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:13.992062 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c9efa6-90cd-4702-95d9-3161d125571f service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:14.001221 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1119/2240] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:31:13 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:31:14.021995 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f8cdd1-0a2a-4b39-bad1-d9847c5f8d55 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:14.094566 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f8cdd1-0a2a-4b39-bad1-d9847c5f8d55 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:14.099388 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6748ee32-5618-4d92-b11d-506489008ae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:14.100266 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6748ee32-5618-4d92-b11d-506489008ae1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:14.100539 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6748ee32-5618-4d92-b11d-506489008ae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:14.100805 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6748ee32-5618-4d92-b11d-506489008ae1 admin admin] PATH_INFO: `/v3/users/b9a877937193493992f96c0bb1f92191` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:14.101975 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6748ee32-5618-4d92-b11d-506489008ae1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9a877937193493992f96c0bb1f92191)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:14.103999 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6748ee32-5618-4d92-b11d-506489008ae1 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:14.254165 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c9efa6-90cd-4702-95d9-3161d125571f service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:14.255683 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-59c9efa6-90cd-4702-95d9-3161d125571f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:14.269775 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c9efa6-90cd-4702-95d9-3161d125571f service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:14.274595 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e597efa-ef28-4552-ab6d-6951805defe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:14.275457 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e597efa-ef28-4552-ab6d-6951805defe6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:14.275787 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e597efa-ef28-4552-ab6d-6951805defe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:14.276029 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e597efa-ef28-4552-ab6d-6951805defe6 admin admin] PATH_INFO: `/v3/users/3e10bbd1be814c9f8216bf64b4128e54` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:14.278300 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e597efa-ef28-4552-ab6d-6951805defe6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e10bbd1be814c9f8216bf64b4128e54)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:14.279214 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e597efa-ef28-4552-ab6d-6951805defe6 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:14.307156 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-6748ee32-5618-4d92-b11d-506489008ae1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9a877937193493992f96c0bb1f92191'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:14.314487 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-6748ee32-5618-4d92-b11d-506489008ae1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9a877937193493992f96c0bb1f92191'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:14.326114 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-6748ee32-5618-4d92-b11d-506489008ae1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9a877937193493992f96c0bb1f92191'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:14.326845 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-6748ee32-5618-4d92-b11d-506489008ae1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9a877937193493992f96c0bb1f92191'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:14.327456 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-6748ee32-5618-4d92-b11d-506489008ae1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9a877937193493992f96c0bb1f92191'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:14.434482 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1122/2241] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 17 19:31:14 2022] DELETE /identity/v3/users/b9a877937193493992f96c0bb1f92191 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:31:14.438815 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4894a19e-4751-4a75-8bdb-5c765ff32efa None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:14.439110 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4894a19e-4751-4a75-8bdb-5c765ff32efa None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:14.439535 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4894a19e-4751-4a75-8bdb-5c765ff32efa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:14.458051 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-4894a19e-4751-4a75-8bdb-5c765ff32efa None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:31:14.465837 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-4894a19e-4751-4a75-8bdb-5c765ff32efa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:14.521941 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1123/2242] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:31:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:31:14.545456 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4894a19e-4751-4a75-8bdb-5c765ff32efa None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:14.546741 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-4894a19e-4751-4a75-8bdb-5c765ff32efa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:14.574064 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-6e597efa-ef28-4552-ab6d-6951805defe6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e10bbd1be814c9f8216bf64b4128e54'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:14.574745 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-6e597efa-ef28-4552-ab6d-6951805defe6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e10bbd1be814c9f8216bf64b4128e54'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:14.586946 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-6e597efa-ef28-4552-ab6d-6951805defe6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e10bbd1be814c9f8216bf64b4128e54'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:14.606575 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4894a19e-4751-4a75-8bdb-5c765ff32efa None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:14.610275 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40566883-1bf1-48db-a0f4-9d1574026b24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:14.610918 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40566883-1bf1-48db-a0f4-9d1574026b24 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:14.611198 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40566883-1bf1-48db-a0f4-9d1574026b24 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:14.611469 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40566883-1bf1-48db-a0f4-9d1574026b24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:14.613241 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-40566883-1bf1-48db-a0f4-9d1574026b24 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:14.624893 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40566883-1bf1-48db-a0f4-9d1574026b24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:14.626330 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40566883-1bf1-48db-a0f4-9d1574026b24 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:14.636255 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1124/2243] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:31:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 17 19:31:14.670994 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-6e597efa-ef28-4552-ab6d-6951805defe6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e10bbd1be814c9f8216bf64b4128e54'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:14.678527 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-6e597efa-ef28-4552-ab6d-6951805defe6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e10bbd1be814c9f8216bf64b4128e54'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:14.685439 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1120/2244] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 17 19:31:14 2022] DELETE /identity/v3/users/3e10bbd1be814c9f8216bf64b4128e54 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 17 19:31:14.690125 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a07bb86-a1ff-4e36-8180-463fe063371b None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:14.690640 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a07bb86-a1ff-4e36-8180-463fe063371b None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:14.690792 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a07bb86-a1ff-4e36-8180-463fe063371b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:14.724637 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-4a07bb86-a1ff-4e36-8180-463fe063371b None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:31:14.724637 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-4a07bb86-a1ff-4e36-8180-463fe063371b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:14.804980 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1125/2245] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:31:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 17 19:31:14.817129 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e597efa-ef28-4552-ab6d-6951805defe6 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:14.818570 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-6e597efa-ef28-4552-ab6d-6951805defe6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:14.938366 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e597efa-ef28-4552-ab6d-6951805defe6 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:14.942114 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82be6f93-8860-4e4f-b348-230eff830b70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:14.942767 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82be6f93-8860-4e4f-b348-230eff830b70 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:14.943048 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82be6f93-8860-4e4f-b348-230eff830b70 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:14.943319 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82be6f93-8860-4e4f-b348-230eff830b70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:14.953367 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82be6f93-8860-4e4f-b348-230eff830b70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:14.954888 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82be6f93-8860-4e4f-b348-230eff830b70 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:14.968954 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1121/2246] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:31:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:31:15.110190 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a07bb86-a1ff-4e36-8180-463fe063371b None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:15.111733 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-4a07bb86-a1ff-4e36-8180-463fe063371b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:15.123655 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a07bb86-a1ff-4e36-8180-463fe063371b None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:15.128480 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a59cc68-8b8d-4b19-b553-2023d5c3021a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:15.129368 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a59cc68-8b8d-4b19-b553-2023d5c3021a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:15.129649 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a59cc68-8b8d-4b19-b553-2023d5c3021a admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:15.130259 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a59cc68-8b8d-4b19-b553-2023d5c3021a admin admin] PATH_INFO: `/v3/projects/66b47f4795be411daf349cf57f8e3a15` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:15.131127 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a59cc68-8b8d-4b19-b553-2023d5c3021a admin admin] RBAC: Authorizing `identity:delete_project(project_id=66b47f4795be411daf349cf57f8e3a15)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:15.133089 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a59cc68-8b8d-4b19-b553-2023d5c3021a admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:15.259161 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-9a59cc68-8b8d-4b19-b553-2023d5c3021a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66b47f4795be411daf349cf57f8e3a15', 'request_id': 'req-9a59cc68-8b8d-4b19-b553-2023d5c3021a'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:15.265852 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-9a59cc68-8b8d-4b19-b553-2023d5c3021a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66b47f4795be411daf349cf57f8e3a15', 'request_id': 'req-9a59cc68-8b8d-4b19-b553-2023d5c3021a'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:15.271283 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-9a59cc68-8b8d-4b19-b553-2023d5c3021a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66b47f4795be411daf349cf57f8e3a15', 'request_id': 'req-9a59cc68-8b8d-4b19-b553-2023d5c3021a'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:15.379305 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-9a59cc68-8b8d-4b19-b553-2023d5c3021a admin admin] The token cache is being invalidate because project 66b47f4795be411daf349cf57f8e3a15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 17 19:31:15.381662 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-9a59cc68-8b8d-4b19-b553-2023d5c3021a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:15.381662 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-9a59cc68-8b8d-4b19-b553-2023d5c3021a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:15.383839 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1126/2247] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 17 19:31:15 2022] DELETE /identity/v3/projects/66b47f4795be411daf349cf57f8e3a15 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 17 19:31:15.388731 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b688ac-3318-43e7-a2d3-526b82e34a51 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:15.389022 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b688ac-3318-43e7-a2d3-526b82e34a51 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:15.389295 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b688ac-3318-43e7-a2d3-526b82e34a51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:15.410388 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-d9b688ac-3318-43e7-a2d3-526b82e34a51 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:31:15.411853 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d9b688ac-3318-43e7-a2d3-526b82e34a51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:15.476948 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1122/2248] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:31:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:31:15.490738 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ab212e-4a1c-4aa7-b2c8-35c816cecad1 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:15.491026 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ab212e-4a1c-4aa7-b2c8-35c816cecad1 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:15.491309 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ab212e-4a1c-4aa7-b2c8-35c816cecad1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:15.513512 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-69ab212e-4a1c-4aa7-b2c8-35c816cecad1 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:31:15.513512 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-69ab212e-4a1c-4aa7-b2c8-35c816cecad1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:15.518871 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1127/2249] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:31:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:31:15.523042 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b688ac-3318-43e7-a2d3-526b82e34a51 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:15.533123 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b688ac-3318-43e7-a2d3-526b82e34a51 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:15.538689 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80f8ecd-38cb-469e-8cd9-ab7ca92c5e00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:15.539393 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80f8ecd-38cb-469e-8cd9-ab7ca92c5e00 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:15.539700 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80f8ecd-38cb-469e-8cd9-ab7ca92c5e00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:15.540143 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80f8ecd-38cb-469e-8cd9-ab7ca92c5e00 admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:15.540860 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80f8ecd-38cb-469e-8cd9-ab7ca92c5e00 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:31:15.541162 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80f8ecd-38cb-469e-8cd9-ab7ca92c5e00 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:15.543342 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80f8ecd-38cb-469e-8cd9-ab7ca92c5e00 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:15.558664 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1123/2250] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 17 19:31:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:31:15.559781 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ab212e-4a1c-4aa7-b2c8-35c816cecad1 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:15.568812 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ab212e-4a1c-4aa7-b2c8-35c816cecad1 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:15.574438 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76ed1a7-05bf-49b0-acaa-31b3e90deb30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:15.574954 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76ed1a7-05bf-49b0-acaa-31b3e90deb30 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:15.575233 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76ed1a7-05bf-49b0-acaa-31b3e90deb30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:15.575789 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76ed1a7-05bf-49b0-acaa-31b3e90deb30 admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:15.576475 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76ed1a7-05bf-49b0-acaa-31b3e90deb30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:15.578427 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76ed1a7-05bf-49b0-acaa-31b3e90deb30 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:15.584236 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80f8ecd-38cb-469e-8cd9-ab7ca92c5e00 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:15.589311 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-f80f8ecd-38cb-469e-8cd9-ab7ca92c5e00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:15.603336 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80f8ecd-38cb-469e-8cd9-ab7ca92c5e00 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:15.613326 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45d40f3-cd4c-43be-b92a-bfe5eb53bd32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:15.613326 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45d40f3-cd4c-43be-b92a-bfe5eb53bd32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:15.613326 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45d40f3-cd4c-43be-b92a-bfe5eb53bd32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:15.613326 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45d40f3-cd4c-43be-b92a-bfe5eb53bd32 admin admin] PATH_INFO: `/v3/projects/14a6f2a77ec74b60815121fc444718c6` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:15.613989 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45d40f3-cd4c-43be-b92a-bfe5eb53bd32 admin admin] RBAC: Authorizing `identity:delete_project(project_id=14a6f2a77ec74b60815121fc444718c6)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:15.613989 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45d40f3-cd4c-43be-b92a-bfe5eb53bd32 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:15.680624 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1128/2251] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 17 19:31:15 2022] POST /identity/v3/projects => generated 389 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:31:15.684925 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76ed1a7-05bf-49b0-acaa-31b3e90deb30 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:15.724958 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c45d40f3-cd4c-43be-b92a-bfe5eb53bd32 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14a6f2a77ec74b60815121fc444718c6', 'request_id': 'req-c45d40f3-cd4c-43be-b92a-bfe5eb53bd32'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:15.730872 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c45d40f3-cd4c-43be-b92a-bfe5eb53bd32 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14a6f2a77ec74b60815121fc444718c6', 'request_id': 'req-c45d40f3-cd4c-43be-b92a-bfe5eb53bd32'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:15.748026 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76ed1a7-05bf-49b0-acaa-31b3e90deb30 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:15.803533 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f153914-7b26-43e5-b7c5-78aa7bc95e8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:15.803940 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f153914-7b26-43e5-b7c5-78aa7bc95e8a admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:15.804215 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f153914-7b26-43e5-b7c5-78aa7bc95e8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:15.804568 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f153914-7b26-43e5-b7c5-78aa7bc95e8a admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:15.805158 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f153914-7b26-43e5-b7c5-78aa7bc95e8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:15.807062 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f153914-7b26-43e5-b7c5-78aa7bc95e8a admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:15.807390 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c45d40f3-cd4c-43be-b92a-bfe5eb53bd32 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14a6f2a77ec74b60815121fc444718c6', 'request_id': 'req-c45d40f3-cd4c-43be-b92a-bfe5eb53bd32'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:15.820835 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c45d40f3-cd4c-43be-b92a-bfe5eb53bd32 admin admin] The token cache is being invalidate because project 14a6f2a77ec74b60815121fc444718c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 17 19:31:15.821117 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c45d40f3-cd4c-43be-b92a-bfe5eb53bd32 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:15.821812 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c45d40f3-cd4c-43be-b92a-bfe5eb53bd32 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:15.823953 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1124/2252] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 17 19:31:15 2022] DELETE /identity/v3/projects/14a6f2a77ec74b60815121fc444718c6 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:31:15.828598 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24664713-6bac-4ada-8367-98125c34cb2f None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:15.828880 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24664713-6bac-4ada-8367-98125c34cb2f None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:15.829152 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24664713-6bac-4ada-8367-98125c34cb2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:15.847308 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-24664713-6bac-4ada-8367-98125c34cb2f None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:31:15.848555 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-24664713-6bac-4ada-8367-98125c34cb2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:15.858168 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1125/2253] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:31:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 17 19:31:15.869311 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5551dc-0eac-4fb0-bc28-887dc4f81f9f None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:15.869704 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5551dc-0eac-4fb0-bc28-887dc4f81f9f None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:15.870277 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5551dc-0eac-4fb0-bc28-887dc4f81f9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:15.888596 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1129/2254] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 17 19:31:15 2022] POST /identity/v3/users => generated 295 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 17 19:31:15.889496 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f153914-7b26-43e5-b7c5-78aa7bc95e8a admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:15.890843 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-9e5551dc-0eac-4fb0-bc28-887dc4f81f9f None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:31:15.891124 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-4f153914-7b26-43e5-b7c5-78aa7bc95e8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:15.892144 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-9e5551dc-0eac-4fb0-bc28-887dc4f81f9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:15.900449 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1126/2255] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:31:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 17 19:31:15.903795 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f153914-7b26-43e5-b7c5-78aa7bc95e8a admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:15.905638 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5551dc-0eac-4fb0-bc28-887dc4f81f9f None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:15.908870 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68025db-9f8f-4d98-93a4-5a4aedab3082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:15.909700 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68025db-9f8f-4d98-93a4-5a4aedab3082 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:15.909871 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68025db-9f8f-4d98-93a4-5a4aedab3082 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:15.910164 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68025db-9f8f-4d98-93a4-5a4aedab3082 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:15.910825 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68025db-9f8f-4d98-93a4-5a4aedab3082 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:15.912442 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68025db-9f8f-4d98-93a4-5a4aedab3082 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:15.914919 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5551dc-0eac-4fb0-bc28-887dc4f81f9f None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:15.920119 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89b9c2a-5d8c-4207-bb98-a09eeb05bc7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:15.920791 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89b9c2a-5d8c-4207-bb98-a09eeb05bc7a admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:15.921067 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89b9c2a-5d8c-4207-bb98-a09eeb05bc7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:15.921340 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89b9c2a-5d8c-4207-bb98-a09eeb05bc7a admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:15.922083 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89b9c2a-5d8c-4207-bb98-a09eeb05bc7a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:31:15.922387 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89b9c2a-5d8c-4207-bb98-a09eeb05bc7a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:15.922966 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1130/2256] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:31:15 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:31:15.924198 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89b9c2a-5d8c-4207-bb98-a09eeb05bc7a admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:15.927251 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68025db-9f8f-4d98-93a4-5a4aedab3082 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:15.936421 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1127/2257] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 17 19:31:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:31:15.937042 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68025db-9f8f-4d98-93a4-5a4aedab3082 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:15.940733 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89b9c2a-5d8c-4207-bb98-a09eeb05bc7a admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:15.942642 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937fa07e-20a4-411e-8439-453c124d1055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:15.943276 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937fa07e-20a4-411e-8439-453c124d1055 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:15.943697 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937fa07e-20a4-411e-8439-453c124d1055 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:15.943993 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937fa07e-20a4-411e-8439-453c124d1055 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:15.944760 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937fa07e-20a4-411e-8439-453c124d1055 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:15.946417 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937fa07e-20a4-411e-8439-453c124d1055 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:15.953954 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89b9c2a-5d8c-4207-bb98-a09eeb05bc7a admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:15.959565 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1131/2258] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:31:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:31:15.959788 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90780b35-f0db-4c2d-8529-8ffbade0309b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:15.960074 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90780b35-f0db-4c2d-8529-8ffbade0309b admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:15.960473 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90780b35-f0db-4c2d-8529-8ffbade0309b admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:15.960827 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90780b35-f0db-4c2d-8529-8ffbade0309b admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:15.961633 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90780b35-f0db-4c2d-8529-8ffbade0309b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:15.962389 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937fa07e-20a4-411e-8439-453c124d1055 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:15.964748 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90780b35-f0db-4c2d-8529-8ffbade0309b admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:15.974114 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937fa07e-20a4-411e-8439-453c124d1055 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:15.979544 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f085f2aa-23bc-4e45-a37b-69fd989ed216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:15.980530 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f085f2aa-23bc-4e45-a37b-69fd989ed216 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:15.980841 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f085f2aa-23bc-4e45-a37b-69fd989ed216 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:15.981180 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f085f2aa-23bc-4e45-a37b-69fd989ed216 admin admin] PATH_INFO: `/v3/projects/fa950f327a934d89a2b848ceedd403e7/users/41831da07b194c6a930a2a3377a7855c/roles/114e6a16e63447d1924707ace5aace25` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:15.996732 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f085f2aa-23bc-4e45-a37b-69fd989ed216 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa950f327a934d89a2b848ceedd403e7, user_id=41831da07b194c6a930a2a3377a7855c, role_id=114e6a16e63447d1924707ace5aace25)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:15.999416 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f085f2aa-23bc-4e45-a37b-69fd989ed216 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:16.083829 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1128/2259] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 17 19:31:15 2022] POST /identity/v3/projects => generated 383 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 17 19:31:16.087758 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90780b35-f0db-4c2d-8529-8ffbade0309b admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:16.105759 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1132/2260] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:31:15 2022] PUT /identity/v3/projects/fa950f327a934d89a2b848ceedd403e7/users/41831da07b194c6a930a2a3377a7855c/roles/114e6a16e63447d1924707ace5aace25 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 17 19:31:16.119815 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808e2b61-9493-4c0f-87e3-7ba9ec8ea031 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:16.121032 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808e2b61-9493-4c0f-87e3-7ba9ec8ea031 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:16.121032 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808e2b61-9493-4c0f-87e3-7ba9ec8ea031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:16.146574 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-808e2b61-9493-4c0f-87e3-7ba9ec8ea031 None None] MFA Rules not processed for user `41831da07b194c6a930a2a3377a7855c`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:31:16.147797 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-808e2b61-9493-4c0f-87e3-7ba9ec8ea031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:16.150664 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90780b35-f0db-4c2d-8529-8ffbade0309b admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:16.205202 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71326383-64f3-4596-979b-207ce0ec1b20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:16.205869 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71326383-64f3-4596-979b-207ce0ec1b20 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:16.206147 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71326383-64f3-4596-979b-207ce0ec1b20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:16.206420 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71326383-64f3-4596-979b-207ce0ec1b20 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:16.207100 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71326383-64f3-4596-979b-207ce0ec1b20 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:16.209060 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71326383-64f3-4596-979b-207ce0ec1b20 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:16.210549 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1133/2261] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:31:16 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 17 19:31:16.222291 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808e2b61-9493-4c0f-87e3-7ba9ec8ea031 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:16.229585 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-808e2b61-9493-4c0f-87e3-7ba9ec8ea031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:16.278958 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808e2b61-9493-4c0f-87e3-7ba9ec8ea031 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:16.282307 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1129/2262] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 17 19:31:16 2022] POST /identity/v3/users => generated 292 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 17 19:31:16.282543 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586d7cdc-1ba3-4106-a115-ed0e5841cd1c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd05409b38ae346458abfe41ebda1c59e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:16.283224 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586d7cdc-1ba3-4106-a115-ed0e5841cd1c service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:16.283532 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586d7cdc-1ba3-4106-a115-ed0e5841cd1c service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:16.283825 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586d7cdc-1ba3-4106-a115-ed0e5841cd1c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:16.286277 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71326383-64f3-4596-979b-207ce0ec1b20 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:16.292545 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586d7cdc-1ba3-4106-a115-ed0e5841cd1c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:16.293966 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586d7cdc-1ba3-4106-a115-ed0e5841cd1c service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:16.296169 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71326383-64f3-4596-979b-207ce0ec1b20 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:16.301118 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc3b260-7674-40bd-83c5-3d676bb9f517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:16.301784 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc3b260-7674-40bd-83c5-3d676bb9f517 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:16.302058 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc3b260-7674-40bd-83c5-3d676bb9f517 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:16.302327 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc3b260-7674-40bd-83c5-3d676bb9f517 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:16.302695 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1134/2263] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:31:16 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:31:16.303158 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc3b260-7674-40bd-83c5-3d676bb9f517 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:16.304964 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc3b260-7674-40bd-83c5-3d676bb9f517 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:16.315654 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1130/2264] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:31:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 17 19:31:16.319747 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586d7cdc-1ba3-4106-a115-ed0e5841cd1c service cinder] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:16.330668 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586d7cdc-1ba3-4106-a115-ed0e5841cd1c service cinder] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:16.335841 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0e57a4-b635-4fc9-952c-88d200b3c4aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:16.335952 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0e57a4-b635-4fc9-952c-88d200b3c4aa admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:16.336157 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0e57a4-b635-4fc9-952c-88d200b3c4aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:16.336435 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0e57a4-b635-4fc9-952c-88d200b3c4aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:16.337145 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0e57a4-b635-4fc9-952c-88d200b3c4aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:16.338802 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0e57a4-b635-4fc9-952c-88d200b3c4aa admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:16.348923 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1135/2265] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:31:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 17 19:31:16.352972 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc3b260-7674-40bd-83c5-3d676bb9f517 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:16.362810 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc3b260-7674-40bd-83c5-3d676bb9f517 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:16.367749 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ce997c-87c6-44d9-9424-a9e9524514cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:16.368626 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ce997c-87c6-44d9-9424-a9e9524514cd admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:16.369157 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ce997c-87c6-44d9-9424-a9e9524514cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:16.369305 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ce997c-87c6-44d9-9424-a9e9524514cd admin admin] PATH_INFO: `/v3/projects/4c80a1d247d44fb9b467b7e312d7ddec/users/af0c3a3e5a934b01aad4d7886bee66b8/roles/114e6a16e63447d1924707ace5aace25` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:16.385427 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ce997c-87c6-44d9-9424-a9e9524514cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c80a1d247d44fb9b467b7e312d7ddec, user_id=af0c3a3e5a934b01aad4d7886bee66b8, role_id=114e6a16e63447d1924707ace5aace25)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:16.388752 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ce997c-87c6-44d9-9424-a9e9524514cd admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:16.457400 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1131/2266] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:31:16 2022] PUT /identity/v3/projects/4c80a1d247d44fb9b467b7e312d7ddec/users/af0c3a3e5a934b01aad4d7886bee66b8/roles/114e6a16e63447d1924707ace5aace25 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 17 19:31:16.463935 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa99782-b5ab-4fc3-b137-6522e4e9e9f3 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:16.464299 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa99782-b5ab-4fc3-b137-6522e4e9e9f3 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:16.464667 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa99782-b5ab-4fc3-b137-6522e4e9e9f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:16.484249 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-dfa99782-b5ab-4fc3-b137-6522e4e9e9f3 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:31:16.485528 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-dfa99782-b5ab-4fc3-b137-6522e4e9e9f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:16.545079 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1136/2267] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:31:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:31:16.554615 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ce997c-87c6-44d9-9424-a9e9524514cd admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:16.556433 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-73ce997c-87c6-44d9-9424-a9e9524514cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:16.615358 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ce997c-87c6-44d9-9424-a9e9524514cd admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:16.620290 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c52091e-60b0-4974-affd-745f5c6a920d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:16.620954 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c52091e-60b0-4974-affd-745f5c6a920d service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:16.621233 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c52091e-60b0-4974-affd-745f5c6a920d service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:16.621506 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c52091e-60b0-4974-affd-745f5c6a920d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:16.631094 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c52091e-60b0-4974-affd-745f5c6a920d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:16.632605 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c52091e-60b0-4974-affd-745f5c6a920d service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:16.643541 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1132/2268] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:31:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:31:18.021500 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddefc5c-a62f-455d-80e0-1df3a7153e40 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:18.022032 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddefc5c-a62f-455d-80e0-1df3a7153e40 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:18.022170 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddefc5c-a62f-455d-80e0-1df3a7153e40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:18.058191 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-eddefc5c-a62f-455d-80e0-1df3a7153e40 None None] MFA Rules not processed for user `af0c3a3e5a934b01aad4d7886bee66b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:31:18.059468 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-eddefc5c-a62f-455d-80e0-1df3a7153e40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:18.137788 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1137/2269] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:31:18 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:31:18.149752 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c52091e-60b0-4974-affd-745f5c6a920d service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:18.207539 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c52091e-60b0-4974-affd-745f5c6a920d service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:18.211246 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca16eb4c-f0cb-4586-9de9-0d9f18126d74 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd05409b38ae346458abfe41ebda1c59e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:18.211929 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca16eb4c-f0cb-4586-9de9-0d9f18126d74 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:18.212206 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca16eb4c-f0cb-4586-9de9-0d9f18126d74 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:18.212474 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca16eb4c-f0cb-4586-9de9-0d9f18126d74 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:18.221590 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca16eb4c-f0cb-4586-9de9-0d9f18126d74 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:18.223284 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca16eb4c-f0cb-4586-9de9-0d9f18126d74 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:18.237585 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1133/2270] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:31:18 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:31:20.030205 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a777ba-1c04-4f82-9a27-958345b3ee1c None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:20.031048 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a777ba-1c04-4f82-9a27-958345b3ee1c None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:20.031327 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a777ba-1c04-4f82-9a27-958345b3ee1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:20.050759 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-41a777ba-1c04-4f82-9a27-958345b3ee1c None None] MFA Rules not processed for user `af0c3a3e5a934b01aad4d7886bee66b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:31:20.052019 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-41a777ba-1c04-4f82-9a27-958345b3ee1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:20.059860 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1138/2271] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:31:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:31:26.596396 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca16eb4c-f0cb-4586-9de9-0d9f18126d74 service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:26.598172 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-ca16eb4c-f0cb-4586-9de9-0d9f18126d74 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:26.678103 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca16eb4c-f0cb-4586-9de9-0d9f18126d74 service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:26.684712 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb79574-fef0-4459-a0c7-5c72b7bbcddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:26.685590 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb79574-fef0-4459-a0c7-5c72b7bbcddd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:26.685866 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb79574-fef0-4459-a0c7-5c72b7bbcddd admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:26.686136 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb79574-fef0-4459-a0c7-5c72b7bbcddd admin admin] PATH_INFO: `/v3/users/9ea19110530a4a5aa030c1567ed975a3` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:26.687404 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb79574-fef0-4459-a0c7-5c72b7bbcddd admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ea19110530a4a5aa030c1567ed975a3)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:26.689412 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb79574-fef0-4459-a0c7-5c72b7bbcddd admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:26.866564 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-abb79574-fef0-4459-a0c7-5c72b7bbcddd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ea19110530a4a5aa030c1567ed975a3'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:26.867251 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-abb79574-fef0-4459-a0c7-5c72b7bbcddd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ea19110530a4a5aa030c1567ed975a3'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:26.879711 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-abb79574-fef0-4459-a0c7-5c72b7bbcddd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ea19110530a4a5aa030c1567ed975a3'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:26.942124 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-abb79574-fef0-4459-a0c7-5c72b7bbcddd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ea19110530a4a5aa030c1567ed975a3'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:26.951388 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-abb79574-fef0-4459-a0c7-5c72b7bbcddd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ea19110530a4a5aa030c1567ed975a3'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:26.958254 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1134/2272] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 17 19:31:26 2022] DELETE /identity/v3/users/9ea19110530a4a5aa030c1567ed975a3 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:31:26.962471 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fca7ce1-5913-4de7-af23-254b60e0c243 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:26.962955 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fca7ce1-5913-4de7-af23-254b60e0c243 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:26.963048 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fca7ce1-5913-4de7-af23-254b60e0c243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:26.990813 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-0fca7ce1-5913-4de7-af23-254b60e0c243 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:31:26.992055 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-0fca7ce1-5913-4de7-af23-254b60e0c243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:27.049939 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1139/2273] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:31:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:31:27.059674 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb79574-fef0-4459-a0c7-5c72b7bbcddd admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:27.061103 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-abb79574-fef0-4459-a0c7-5c72b7bbcddd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:27.156997 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb79574-fef0-4459-a0c7-5c72b7bbcddd admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:27.160770 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b3e4b0-82a2-46b1-adbf-651b2894a4ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:27.161437 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b3e4b0-82a2-46b1-adbf-651b2894a4ca service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:27.161710 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b3e4b0-82a2-46b1-adbf-651b2894a4ca service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:27.161977 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b3e4b0-82a2-46b1-adbf-651b2894a4ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:27.171629 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b3e4b0-82a2-46b1-adbf-651b2894a4ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:27.173060 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b3e4b0-82a2-46b1-adbf-651b2894a4ca service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:27.182776 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1135/2274] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:31:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:31:27.805736 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fca7ce1-5913-4de7-af23-254b60e0c243 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:27.807208 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-0fca7ce1-5913-4de7-af23-254b60e0c243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:27.819113 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fca7ce1-5913-4de7-af23-254b60e0c243 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:27.823816 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49175581-5aa6-430a-b7bd-75278b787ba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:27.825158 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49175581-5aa6-430a-b7bd-75278b787ba5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:27.825433 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49175581-5aa6-430a-b7bd-75278b787ba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:27.825701 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49175581-5aa6-430a-b7bd-75278b787ba5 admin admin] PATH_INFO: `/v3/projects/e84051f21083467db3cb5bb13d8950e0` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:27.826905 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49175581-5aa6-430a-b7bd-75278b787ba5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e84051f21083467db3cb5bb13d8950e0)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:27.829021 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49175581-5aa6-430a-b7bd-75278b787ba5 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:27.954563 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-49175581-5aa6-430a-b7bd-75278b787ba5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e84051f21083467db3cb5bb13d8950e0', 'request_id': 'req-49175581-5aa6-430a-b7bd-75278b787ba5'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:27.960612 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-49175581-5aa6-430a-b7bd-75278b787ba5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e84051f21083467db3cb5bb13d8950e0', 'request_id': 'req-49175581-5aa6-430a-b7bd-75278b787ba5'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:27.966545 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-49175581-5aa6-430a-b7bd-75278b787ba5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e84051f21083467db3cb5bb13d8950e0', 'request_id': 'req-49175581-5aa6-430a-b7bd-75278b787ba5'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:28.121853 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-49175581-5aa6-430a-b7bd-75278b787ba5 admin admin] The token cache is being invalidate because project e84051f21083467db3cb5bb13d8950e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 17 19:31:28.122163 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-49175581-5aa6-430a-b7bd-75278b787ba5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:28.122768 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-49175581-5aa6-430a-b7bd-75278b787ba5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:28.124378 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1140/2275] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 17 19:31:27 2022] DELETE /identity/v3/projects/e84051f21083467db3cb5bb13d8950e0 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:31:28.129189 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd519cf2-3bd1-488c-9ebb-7f46e8e160c9 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:28.129658 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd519cf2-3bd1-488c-9ebb-7f46e8e160c9 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:28.129776 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd519cf2-3bd1-488c-9ebb-7f46e8e160c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:28.166768 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-fd519cf2-3bd1-488c-9ebb-7f46e8e160c9 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:31:28.168039 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-fd519cf2-3bd1-488c-9ebb-7f46e8e160c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:28.226293 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1136/2276] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:31:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:31:28.235243 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d65dd62-2c44-46ce-9b13-75fcf5e025b5 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:28.235548 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d65dd62-2c44-46ce-9b13-75fcf5e025b5 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:28.235821 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d65dd62-2c44-46ce-9b13-75fcf5e025b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:28.253917 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-2d65dd62-2c44-46ce-9b13-75fcf5e025b5 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:31:28.255086 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-2d65dd62-2c44-46ce-9b13-75fcf5e025b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:28.262900 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1141/2277] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:31:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:31:28.266908 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd519cf2-3bd1-488c-9ebb-7f46e8e160c9 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:28.276204 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd519cf2-3bd1-488c-9ebb-7f46e8e160c9 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:28.281174 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee602cea-6603-406e-a9e5-c8cdf700abf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:28.281836 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee602cea-6603-406e-a9e5-c8cdf700abf8 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:28.282108 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee602cea-6603-406e-a9e5-c8cdf700abf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:28.282374 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee602cea-6603-406e-a9e5-c8cdf700abf8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:28.283082 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee602cea-6603-406e-a9e5-c8cdf700abf8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:31:28.283379 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee602cea-6603-406e-a9e5-c8cdf700abf8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:28.284993 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee602cea-6603-406e-a9e5-c8cdf700abf8 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:28.296816 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1137/2278] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 17 19:31:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:31:28.300807 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d65dd62-2c44-46ce-9b13-75fcf5e025b5 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:28.309672 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d65dd62-2c44-46ce-9b13-75fcf5e025b5 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:28.314349 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b44efe5-51f6-4158-ae10-827420a9cff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:28.314953 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b44efe5-51f6-4158-ae10-827420a9cff7 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:28.315228 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b44efe5-51f6-4158-ae10-827420a9cff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:28.315526 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b44efe5-51f6-4158-ae10-827420a9cff7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:28.316210 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b44efe5-51f6-4158-ae10-827420a9cff7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:28.318065 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b44efe5-51f6-4158-ae10-827420a9cff7 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:28.446358 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1142/2279] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 17 19:31:28 2022] POST /identity/v3/projects => generated 385 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 17 19:31:28.451652 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee602cea-6603-406e-a9e5-c8cdf700abf8 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:28.461054 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee602cea-6603-406e-a9e5-c8cdf700abf8 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:28.525326 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27db323-6fd4-475d-9da0-4514a2b5a1b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:28.525998 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27db323-6fd4-475d-9da0-4514a2b5a1b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:28.526275 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27db323-6fd4-475d-9da0-4514a2b5a1b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:28.526546 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27db323-6fd4-475d-9da0-4514a2b5a1b9 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:28.527914 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27db323-6fd4-475d-9da0-4514a2b5a1b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:28.529869 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27db323-6fd4-475d-9da0-4514a2b5a1b9 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:28.613391 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1138/2280] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 17 19:31:28 2022] POST /identity/v3/users => generated 293 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:31:28.617392 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b44efe5-51f6-4158-ae10-827420a9cff7 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:28.627678 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b44efe5-51f6-4158-ae10-827420a9cff7 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:28.632358 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bc00b8-97b0-434a-9225-62d951a2fa63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:28.633002 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bc00b8-97b0-434a-9225-62d951a2fa63 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:28.633301 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bc00b8-97b0-434a-9225-62d951a2fa63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:28.633558 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bc00b8-97b0-434a-9225-62d951a2fa63 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:28.634277 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bc00b8-97b0-434a-9225-62d951a2fa63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:28.635939 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bc00b8-97b0-434a-9225-62d951a2fa63 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:28.646909 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1143/2281] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:31:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:31:28.650826 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27db323-6fd4-475d-9da0-4514a2b5a1b9 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:28.661433 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27db323-6fd4-475d-9da0-4514a2b5a1b9 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:28.666061 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dedc51-d3e2-4c3c-afac-63fbd2c68370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:28.666705 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dedc51-d3e2-4c3c-afac-63fbd2c68370 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:28.666984 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dedc51-d3e2-4c3c-afac-63fbd2c68370 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:28.667253 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dedc51-d3e2-4c3c-afac-63fbd2c68370 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:28.667993 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dedc51-d3e2-4c3c-afac-63fbd2c68370 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:28.669577 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dedc51-d3e2-4c3c-afac-63fbd2c68370 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:28.680243 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1139/2282] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:31:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:31:28.689479 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bc00b8-97b0-434a-9225-62d951a2fa63 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:28.699793 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bc00b8-97b0-434a-9225-62d951a2fa63 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:28.704695 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dde15e-da7e-452b-94ac-97df27ec1973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:28.705560 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dde15e-da7e-452b-94ac-97df27ec1973 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:28.705839 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dde15e-da7e-452b-94ac-97df27ec1973 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:28.706118 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dde15e-da7e-452b-94ac-97df27ec1973 admin admin] PATH_INFO: `/v3/projects/37cad1bf9c954fd3b4736d5823dce622/users/11afb29192e04ac3added23ac5cae56e/roles/114e6a16e63447d1924707ace5aace25` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:28.721069 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dde15e-da7e-452b-94ac-97df27ec1973 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37cad1bf9c954fd3b4736d5823dce622, user_id=11afb29192e04ac3added23ac5cae56e, role_id=114e6a16e63447d1924707ace5aace25)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:28.724068 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dde15e-da7e-452b-94ac-97df27ec1973 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:28.791110 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1144/2283] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:31:28 2022] PUT /identity/v3/projects/37cad1bf9c954fd3b4736d5823dce622/users/11afb29192e04ac3added23ac5cae56e/roles/114e6a16e63447d1924707ace5aace25 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:31:28.803958 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f15294-3d5e-4792-ae7b-3bdbfaa656d5 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:28.804246 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f15294-3d5e-4792-ae7b-3bdbfaa656d5 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:28.804523 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f15294-3d5e-4792-ae7b-3bdbfaa656d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:28.831315 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-66f15294-3d5e-4792-ae7b-3bdbfaa656d5 None None] MFA Rules not processed for user `11afb29192e04ac3added23ac5cae56e`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:31:28.832541 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-66f15294-3d5e-4792-ae7b-3bdbfaa656d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:28.894440 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1140/2284] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:31:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:31:28.898862 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dde15e-da7e-452b-94ac-97df27ec1973 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:28.960437 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dde15e-da7e-452b-94ac-97df27ec1973 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:28.965340 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df041ed-8ea8-4748-9fbb-62a6b83301d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:28.965983 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df041ed-8ea8-4748-9fbb-62a6b83301d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:28.966256 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df041ed-8ea8-4748-9fbb-62a6b83301d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:28.966524 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df041ed-8ea8-4748-9fbb-62a6b83301d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:28.967209 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df041ed-8ea8-4748-9fbb-62a6b83301d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:28.969163 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df041ed-8ea8-4748-9fbb-62a6b83301d8 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:29.051746 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1145/2285] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 17 19:31:28 2022] POST /identity/v3/projects => generated 383 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:31:29.055744 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f15294-3d5e-4792-ae7b-3bdbfaa656d5 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:29.066832 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f15294-3d5e-4792-ae7b-3bdbfaa656d5 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:29.121331 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e5bcd3-4c49-41d7-a046-45bcf1eea965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:29.121984 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e5bcd3-4c49-41d7-a046-45bcf1eea965 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:29.122271 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e5bcd3-4c49-41d7-a046-45bcf1eea965 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:29.122569 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e5bcd3-4c49-41d7-a046-45bcf1eea965 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:29.123271 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e5bcd3-4c49-41d7-a046-45bcf1eea965 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:29.125341 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e5bcd3-4c49-41d7-a046-45bcf1eea965 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:29.210284 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1141/2286] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 17 19:31:29 2022] POST /identity/v3/users => generated 292 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:31:29.214224 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df041ed-8ea8-4748-9fbb-62a6b83301d8 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:29.224650 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df041ed-8ea8-4748-9fbb-62a6b83301d8 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:29.229189 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ba328c-626d-4f5e-9a29-b7277a0e4c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:29.229825 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ba328c-626d-4f5e-9a29-b7277a0e4c4c admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:29.230600 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ba328c-626d-4f5e-9a29-b7277a0e4c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:29.230873 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ba328c-626d-4f5e-9a29-b7277a0e4c4c admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:29.231642 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ba328c-626d-4f5e-9a29-b7277a0e4c4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:29.233306 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ba328c-626d-4f5e-9a29-b7277a0e4c4c admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:29.243996 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1146/2287] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:31:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:31:29.248504 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e5bcd3-4c49-41d7-a046-45bcf1eea965 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:29.258812 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e5bcd3-4c49-41d7-a046-45bcf1eea965 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:29.263615 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7512d7-24da-4b93-bb33-1c58fa15ef5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:29.264270 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7512d7-24da-4b93-bb33-1c58fa15ef5c admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:29.264549 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7512d7-24da-4b93-bb33-1c58fa15ef5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:29.264822 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7512d7-24da-4b93-bb33-1c58fa15ef5c admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:29.265572 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7512d7-24da-4b93-bb33-1c58fa15ef5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:29.267175 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7512d7-24da-4b93-bb33-1c58fa15ef5c admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:29.277168 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1142/2288] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:31:29 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:31:29.281117 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ba328c-626d-4f5e-9a29-b7277a0e4c4c admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:29.291337 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ba328c-626d-4f5e-9a29-b7277a0e4c4c admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:29.296770 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27a5c33-70fb-4da4-afc7-cbb5527acf24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:29.297639 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27a5c33-70fb-4da4-afc7-cbb5527acf24 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:29.297912 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27a5c33-70fb-4da4-afc7-cbb5527acf24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:29.298200 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27a5c33-70fb-4da4-afc7-cbb5527acf24 admin admin] PATH_INFO: `/v3/projects/d877e10950e8462fbf52da32de0a502a/users/63d29b10bc7f4cde9c0ad3523b0380e9/roles/114e6a16e63447d1924707ace5aace25` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:29.313122 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27a5c33-70fb-4da4-afc7-cbb5527acf24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d877e10950e8462fbf52da32de0a502a, user_id=63d29b10bc7f4cde9c0ad3523b0380e9, role_id=114e6a16e63447d1924707ace5aace25)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:29.315835 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27a5c33-70fb-4da4-afc7-cbb5527acf24 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:29.393257 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1147/2289] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:31:29 2022] PUT /identity/v3/projects/d877e10950e8462fbf52da32de0a502a/users/63d29b10bc7f4cde9c0ad3523b0380e9/roles/114e6a16e63447d1924707ace5aace25 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:31:29.404367 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efbd80f-6315-47d0-b1ca-66ebd9331409 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:29.406055 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efbd80f-6315-47d0-b1ca-66ebd9331409 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:29.406055 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efbd80f-6315-47d0-b1ca-66ebd9331409 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:29.431448 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-6efbd80f-6315-47d0-b1ca-66ebd9331409 None None] MFA Rules not processed for user `63d29b10bc7f4cde9c0ad3523b0380e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:31:29.432673 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-6efbd80f-6315-47d0-b1ca-66ebd9331409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:29.503031 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1143/2290] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:31:29 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:31:29.507328 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27a5c33-70fb-4da4-afc7-cbb5527acf24 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:29.571079 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27a5c33-70fb-4da4-afc7-cbb5527acf24 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:29.577530 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51deddc-ab96-48db-a1dd-a899575b8840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:29.577979 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51deddc-ab96-48db-a1dd-a899575b8840 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:29.578257 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51deddc-ab96-48db-a1dd-a899575b8840 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:29.578531 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51deddc-ab96-48db-a1dd-a899575b8840 admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:29.579210 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51deddc-ab96-48db-a1dd-a899575b8840 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:29.581141 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51deddc-ab96-48db-a1dd-a899575b8840 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:29.703250 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1148/2291] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 17 19:31:29 2022] POST /identity/v3/projects => generated 383 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:31:29.712147 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efbd80f-6315-47d0-b1ca-66ebd9331409 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:29.723368 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efbd80f-6315-47d0-b1ca-66ebd9331409 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:29.778572 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96332d1f-8b1a-4d76-833d-77ac2ddd4eff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:29.779228 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96332d1f-8b1a-4d76-833d-77ac2ddd4eff admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:29.779533 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96332d1f-8b1a-4d76-833d-77ac2ddd4eff admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:29.779806 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96332d1f-8b1a-4d76-833d-77ac2ddd4eff admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:29.780471 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96332d1f-8b1a-4d76-833d-77ac2ddd4eff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:29.782408 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96332d1f-8b1a-4d76-833d-77ac2ddd4eff admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:29.860118 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1144/2292] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 17 19:31:29 2022] POST /identity/v3/users => generated 298 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:31:29.868305 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51deddc-ab96-48db-a1dd-a899575b8840 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:29.879265 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51deddc-ab96-48db-a1dd-a899575b8840 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:29.884483 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8836af47-39ff-4d33-b42f-4a21e95836c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:29.885145 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8836af47-39ff-4d33-b42f-4a21e95836c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:29.885426 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8836af47-39ff-4d33-b42f-4a21e95836c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:29.885699 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8836af47-39ff-4d33-b42f-4a21e95836c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:29.886423 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8836af47-39ff-4d33-b42f-4a21e95836c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:29.888052 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8836af47-39ff-4d33-b42f-4a21e95836c8 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:29.898683 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1149/2293] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:31:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:31:29.903280 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96332d1f-8b1a-4d76-833d-77ac2ddd4eff admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:29.913072 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96332d1f-8b1a-4d76-833d-77ac2ddd4eff admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:29.917746 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb46cfc-ddb8-465c-8906-fa1490dd041a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:29.919211 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb46cfc-ddb8-465c-8906-fa1490dd041a admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:29.919498 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb46cfc-ddb8-465c-8906-fa1490dd041a admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:29.919810 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb46cfc-ddb8-465c-8906-fa1490dd041a admin admin] PATH_INFO: `/v3/projects/913af32bc6644584b35aa5c1ae6ffabb/users/3333e72e51874b6fa9bb9d673d0a4a34/roles/60af5088793e4379b88821a86552a344` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:29.934375 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb46cfc-ddb8-465c-8906-fa1490dd041a admin admin] RBAC: Authorizing `identity:create_grant(project_id=913af32bc6644584b35aa5c1ae6ffabb, user_id=3333e72e51874b6fa9bb9d673d0a4a34, role_id=60af5088793e4379b88821a86552a344)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:29.937187 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb46cfc-ddb8-465c-8906-fa1490dd041a admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:30.022237 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1145/2294] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:31:29 2022] PUT /identity/v3/projects/913af32bc6644584b35aa5c1ae6ffabb/users/3333e72e51874b6fa9bb9d673d0a4a34/roles/60af5088793e4379b88821a86552a344 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:31:30.026221 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8836af47-39ff-4d33-b42f-4a21e95836c8 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:30.036501 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8836af47-39ff-4d33-b42f-4a21e95836c8 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:30.090606 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb5166c-53fa-4ec6-b2d4-53a99b7beeab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:30.091249 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb5166c-53fa-4ec6-b2d4-53a99b7beeab admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:30.091612 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb5166c-53fa-4ec6-b2d4-53a99b7beeab admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:30.091885 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb5166c-53fa-4ec6-b2d4-53a99b7beeab admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:30.092591 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb5166c-53fa-4ec6-b2d4-53a99b7beeab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:30.094171 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb5166c-53fa-4ec6-b2d4-53a99b7beeab admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:30.103807 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1150/2295] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:31:30 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:31:30.107897 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb46cfc-ddb8-465c-8906-fa1490dd041a admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:30.117938 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb46cfc-ddb8-465c-8906-fa1490dd041a admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:30.122509 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e7cc6d-501e-4cc2-a411-edd71d309e48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:30.123364 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e7cc6d-501e-4cc2-a411-edd71d309e48 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:30.123670 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e7cc6d-501e-4cc2-a411-edd71d309e48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:30.123941 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e7cc6d-501e-4cc2-a411-edd71d309e48 admin admin] PATH_INFO: `/v3/projects/913af32bc6644584b35aa5c1ae6ffabb/users/3333e72e51874b6fa9bb9d673d0a4a34/roles/60af5088793e4379b88821a86552a344` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:30.125866 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e7cc6d-501e-4cc2-a411-edd71d309e48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=913af32bc6644584b35aa5c1ae6ffabb, user_id=3333e72e51874b6fa9bb9d673d0a4a34, role_id=60af5088793e4379b88821a86552a344)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:30.128539 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e7cc6d-501e-4cc2-a411-edd71d309e48 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:30.138770 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1146/2296] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:31:30 2022] PUT /identity/v3/projects/913af32bc6644584b35aa5c1ae6ffabb/users/3333e72e51874b6fa9bb9d673d0a4a34/roles/60af5088793e4379b88821a86552a344 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 17 19:31:30.148458 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40751ba0-4637-41d2-a8ed-4af785e5b111 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:30.148749 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40751ba0-4637-41d2-a8ed-4af785e5b111 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:30.149025 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40751ba0-4637-41d2-a8ed-4af785e5b111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:30.174842 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-40751ba0-4637-41d2-a8ed-4af785e5b111 None None] MFA Rules not processed for user `3333e72e51874b6fa9bb9d673d0a4a34`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:31:30.176058 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-40751ba0-4637-41d2-a8ed-4af785e5b111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:30.239617 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1151/2297] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:31:30 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 17 19:31:30.250875 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e7cc6d-501e-4cc2-a411-edd71d309e48 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:30.252688 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-88e7cc6d-501e-4cc2-a411-edd71d309e48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:30.307052 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e7cc6d-501e-4cc2-a411-edd71d309e48 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:30.311111 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1831fc-dd10-46c4-83cf-0d9c6a647d84 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd05409b38ae346458abfe41ebda1c59e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:30.311781 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1831fc-dd10-46c4-83cf-0d9c6a647d84 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:30.312061 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1831fc-dd10-46c4-83cf-0d9c6a647d84 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:30.312336 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1831fc-dd10-46c4-83cf-0d9c6a647d84 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:30.370177 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1831fc-dd10-46c4-83cf-0d9c6a647d84 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:30.370177 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1831fc-dd10-46c4-83cf-0d9c6a647d84 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:30.378156 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1147/2298] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:31:30 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:31:32.420918 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40751ba0-4637-41d2-a8ed-4af785e5b111 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:32.430258 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40751ba0-4637-41d2-a8ed-4af785e5b111 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:32.433887 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b78529d-6b7a-4c97-96f0-1e67335412e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd05409b38ae346458abfe41ebda1c59e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:32.434515 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b78529d-6b7a-4c97-96f0-1e67335412e9 service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:32.434798 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b78529d-6b7a-4c97-96f0-1e67335412e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:32.435076 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b78529d-6b7a-4c97-96f0-1e67335412e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:32.500776 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b78529d-6b7a-4c97-96f0-1e67335412e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:32.500776 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b78529d-6b7a-4c97-96f0-1e67335412e9 service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:32.503352 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1152/2299] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:31:32 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:31:33.119006 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1831fc-dd10-46c4-83cf-0d9c6a647d84 service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:33.120432 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-6d1831fc-dd10-46c4-83cf-0d9c6a647d84 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:33.185546 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1831fc-dd10-46c4-83cf-0d9c6a647d84 service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:33.191941 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9128cc19-dedd-4e11-9d31-46a9b18d9aa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:33.192819 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9128cc19-dedd-4e11-9d31-46a9b18d9aa7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:33.193099 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9128cc19-dedd-4e11-9d31-46a9b18d9aa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:33.193392 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9128cc19-dedd-4e11-9d31-46a9b18d9aa7 admin admin] PATH_INFO: `/v3/users/8d88717e79ef4ce8802e9c41ac20142b` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:33.194733 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9128cc19-dedd-4e11-9d31-46a9b18d9aa7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d88717e79ef4ce8802e9c41ac20142b)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:33.196835 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9128cc19-dedd-4e11-9d31-46a9b18d9aa7 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:33.402090 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-9128cc19-dedd-4e11-9d31-46a9b18d9aa7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d88717e79ef4ce8802e9c41ac20142b'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:33.402778 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-9128cc19-dedd-4e11-9d31-46a9b18d9aa7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d88717e79ef4ce8802e9c41ac20142b'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:33.414848 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-9128cc19-dedd-4e11-9d31-46a9b18d9aa7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d88717e79ef4ce8802e9c41ac20142b'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:33.484844 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-9128cc19-dedd-4e11-9d31-46a9b18d9aa7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d88717e79ef4ce8802e9c41ac20142b'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:33.492044 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-9128cc19-dedd-4e11-9d31-46a9b18d9aa7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d88717e79ef4ce8802e9c41ac20142b'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:33.498310 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1148/2300] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 17 19:31:33 2022] DELETE /identity/v3/users/8d88717e79ef4ce8802e9c41ac20142b => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:31:33.505282 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc748a67-4d84-4ca1-949a-eb1aa1849d34 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:33.505740 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc748a67-4d84-4ca1-949a-eb1aa1849d34 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:33.505853 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc748a67-4d84-4ca1-949a-eb1aa1849d34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:33.523075 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-fc748a67-4d84-4ca1-949a-eb1aa1849d34 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:31:33.524302 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-fc748a67-4d84-4ca1-949a-eb1aa1849d34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:33.581514 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1153/2301] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:31:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 17 19:31:33.591075 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9128cc19-dedd-4e11-9d31-46a9b18d9aa7 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:33.592463 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-9128cc19-dedd-4e11-9d31-46a9b18d9aa7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:33.647449 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9128cc19-dedd-4e11-9d31-46a9b18d9aa7 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:33.651086 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea320c2-db37-4d4f-90c4-149afbf3ebde service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:33.651749 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea320c2-db37-4d4f-90c4-149afbf3ebde service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:33.652061 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea320c2-db37-4d4f-90c4-149afbf3ebde service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:33.652333 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea320c2-db37-4d4f-90c4-149afbf3ebde service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:33.661270 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea320c2-db37-4d4f-90c4-149afbf3ebde service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:33.662692 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea320c2-db37-4d4f-90c4-149afbf3ebde service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:33.672001 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1149/2302] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:31:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:31:34.286350 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc748a67-4d84-4ca1-949a-eb1aa1849d34 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:34.287799 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-fc748a67-4d84-4ca1-949a-eb1aa1849d34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:34.300643 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc748a67-4d84-4ca1-949a-eb1aa1849d34 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:34.306060 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b31512c-33f5-4486-88b6-5cd4235f6beb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:34.306912 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b31512c-33f5-4486-88b6-5cd4235f6beb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:34.307188 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b31512c-33f5-4486-88b6-5cd4235f6beb admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:34.307459 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b31512c-33f5-4486-88b6-5cd4235f6beb admin admin] PATH_INFO: `/v3/projects/def5a0bf5f2843feafabda0ba4b00b72` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:34.308757 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b31512c-33f5-4486-88b6-5cd4235f6beb admin admin] RBAC: Authorizing `identity:delete_project(project_id=def5a0bf5f2843feafabda0ba4b00b72)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:34.310822 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b31512c-33f5-4486-88b6-5cd4235f6beb admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:34.425996 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-9b31512c-33f5-4486-88b6-5cd4235f6beb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'def5a0bf5f2843feafabda0ba4b00b72', 'request_id': 'req-9b31512c-33f5-4486-88b6-5cd4235f6beb'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:34.432339 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-9b31512c-33f5-4486-88b6-5cd4235f6beb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'def5a0bf5f2843feafabda0ba4b00b72', 'request_id': 'req-9b31512c-33f5-4486-88b6-5cd4235f6beb'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:34.442784 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-9b31512c-33f5-4486-88b6-5cd4235f6beb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'def5a0bf5f2843feafabda0ba4b00b72', 'request_id': 'req-9b31512c-33f5-4486-88b6-5cd4235f6beb'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:34.536344 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-9b31512c-33f5-4486-88b6-5cd4235f6beb admin admin] The token cache is being invalidate because project def5a0bf5f2843feafabda0ba4b00b72 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 17 19:31:34.537062 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-9b31512c-33f5-4486-88b6-5cd4235f6beb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:34.537406 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-9b31512c-33f5-4486-88b6-5cd4235f6beb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:34.538966 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1154/2303] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 17 19:31:34 2022] DELETE /identity/v3/projects/def5a0bf5f2843feafabda0ba4b00b72 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:31:34.543262 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea320c2-db37-4d4f-90c4-149afbf3ebde service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:34.553100 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea320c2-db37-4d4f-90c4-149afbf3ebde service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:34.620538 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845cab3d-75ef-4ab0-a50f-d36404b269e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:34.621420 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845cab3d-75ef-4ab0-a50f-d36404b269e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:34.621726 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845cab3d-75ef-4ab0-a50f-d36404b269e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:34.622003 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845cab3d-75ef-4ab0-a50f-d36404b269e6 admin admin] PATH_INFO: `/v3/users/d129b5884ac74d9f90a74dcdfc82678f` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:34.623351 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845cab3d-75ef-4ab0-a50f-d36404b269e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d129b5884ac74d9f90a74dcdfc82678f)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:34.625333 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845cab3d-75ef-4ab0-a50f-d36404b269e6 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:34.846812 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-845cab3d-75ef-4ab0-a50f-d36404b269e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd129b5884ac74d9f90a74dcdfc82678f'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:34.847778 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-845cab3d-75ef-4ab0-a50f-d36404b269e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd129b5884ac74d9f90a74dcdfc82678f'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:34.859203 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-845cab3d-75ef-4ab0-a50f-d36404b269e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd129b5884ac74d9f90a74dcdfc82678f'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:34.953475 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-845cab3d-75ef-4ab0-a50f-d36404b269e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd129b5884ac74d9f90a74dcdfc82678f'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:34.963140 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-845cab3d-75ef-4ab0-a50f-d36404b269e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd129b5884ac74d9f90a74dcdfc82678f'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:34.967272 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1150/2304] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 17 19:31:34 2022] DELETE /identity/v3/users/d129b5884ac74d9f90a74dcdfc82678f => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:31:35.440804 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b31512c-33f5-4486-88b6-5cd4235f6beb admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:35.442310 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-9b31512c-33f5-4486-88b6-5cd4235f6beb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:35.465639 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b31512c-33f5-4486-88b6-5cd4235f6beb admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:35.525379 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d3a6bf-43a6-4753-972e-49a7640e3b8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:35.526240 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d3a6bf-43a6-4753-972e-49a7640e3b8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:35.526524 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d3a6bf-43a6-4753-972e-49a7640e3b8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:35.526801 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d3a6bf-43a6-4753-972e-49a7640e3b8e admin admin] PATH_INFO: `/v3/projects/8d28e5936a9a42cdb5f921e4a6c947f8` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:35.528076 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d3a6bf-43a6-4753-972e-49a7640e3b8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d28e5936a9a42cdb5f921e4a6c947f8)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:35.530029 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d3a6bf-43a6-4753-972e-49a7640e3b8e admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:35.671717 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-95d3a6bf-43a6-4753-972e-49a7640e3b8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d28e5936a9a42cdb5f921e4a6c947f8', 'request_id': 'req-95d3a6bf-43a6-4753-972e-49a7640e3b8e'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:35.679150 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-95d3a6bf-43a6-4753-972e-49a7640e3b8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d28e5936a9a42cdb5f921e4a6c947f8', 'request_id': 'req-95d3a6bf-43a6-4753-972e-49a7640e3b8e'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:35.685310 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-95d3a6bf-43a6-4753-972e-49a7640e3b8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d28e5936a9a42cdb5f921e4a6c947f8', 'request_id': 'req-95d3a6bf-43a6-4753-972e-49a7640e3b8e'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:35.741463 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845cab3d-75ef-4ab0-a50f-d36404b269e6 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:35.745896 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-845cab3d-75ef-4ab0-a50f-d36404b269e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:35.800895 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845cab3d-75ef-4ab0-a50f-d36404b269e6 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:35.816791 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-95d3a6bf-43a6-4753-972e-49a7640e3b8e admin admin] The token cache is being invalidate because project 8d28e5936a9a42cdb5f921e4a6c947f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 17 19:31:35.820010 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-95d3a6bf-43a6-4753-972e-49a7640e3b8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:35.820010 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-95d3a6bf-43a6-4753-972e-49a7640e3b8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:35.820010 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1155/2305] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 17 19:31:35 2022] DELETE /identity/v3/projects/8d28e5936a9a42cdb5f921e4a6c947f8 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:31:35.831261 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f338d8b-d750-4b62-ba6d-baefe97d5515 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:35.831261 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f338d8b-d750-4b62-ba6d-baefe97d5515 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:35.831261 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f338d8b-d750-4b62-ba6d-baefe97d5515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:35.853794 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-4f338d8b-d750-4b62-ba6d-baefe97d5515 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:31:35.855882 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-4f338d8b-d750-4b62-ba6d-baefe97d5515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:35.856016 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce5aad2-0fea-43b6-b30e-a275a26804a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd05409b38ae346458abfe41ebda1c59e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:35.856135 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce5aad2-0fea-43b6-b30e-a275a26804a4 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:35.860732 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce5aad2-0fea-43b6-b30e-a275a26804a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:35.860732 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce5aad2-0fea-43b6-b30e-a275a26804a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:35.861971 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-3ce5aad2-0fea-43b6-b30e-a275a26804a4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:35.923772 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1156/2306] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:31:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:31:35.927736 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce5aad2-0fea-43b6-b30e-a275a26804a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:35.929329 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce5aad2-0fea-43b6-b30e-a275a26804a4 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:35.931315 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc361dcf-5d49-4b6d-9eab-f78b5e7ab83d None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:35.931636 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc361dcf-5d49-4b6d-9eab-f78b5e7ab83d None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:35.931909 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc361dcf-5d49-4b6d-9eab-f78b5e7ab83d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:35.940390 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1151/2307] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:31:35 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:31:35.950945 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-fc361dcf-5d49-4b6d-9eab-f78b5e7ab83d None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:31:35.952220 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-fc361dcf-5d49-4b6d-9eab-f78b5e7ab83d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:35.960414 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1157/2308] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:31:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 17 19:31:35.964992 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce5aad2-0fea-43b6-b30e-a275a26804a4 service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:35.975364 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce5aad2-0fea-43b6-b30e-a275a26804a4 service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:35.980766 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ac5ef2-1a2e-4961-a823-46f22a2636e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:35.981369 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ac5ef2-1a2e-4961-a823-46f22a2636e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:35.981646 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ac5ef2-1a2e-4961-a823-46f22a2636e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:35.981918 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ac5ef2-1a2e-4961-a823-46f22a2636e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:35.982647 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ac5ef2-1a2e-4961-a823-46f22a2636e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:31:35.982949 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ac5ef2-1a2e-4961-a823-46f22a2636e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:35.984595 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ac5ef2-1a2e-4961-a823-46f22a2636e0 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:35.997522 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1152/2309] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 17 19:31:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:31:36.002064 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc361dcf-5d49-4b6d-9eab-f78b5e7ab83d None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:36.011400 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc361dcf-5d49-4b6d-9eab-f78b5e7ab83d None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:36.017344 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a8d04b-3689-4edf-bcc9-dcf07b505262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:36.020756 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a8d04b-3689-4edf-bcc9-dcf07b505262 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:36.020756 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a8d04b-3689-4edf-bcc9-dcf07b505262 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:36.020756 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a8d04b-3689-4edf-bcc9-dcf07b505262 admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:36.020756 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a8d04b-3689-4edf-bcc9-dcf07b505262 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:36.021164 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a8d04b-3689-4edf-bcc9-dcf07b505262 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:36.122910 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1158/2310] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 17 19:31:36 2022] POST /identity/v3/projects => generated 373 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:31:36.123979 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ac5ef2-1a2e-4961-a823-46f22a2636e0 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:36.134372 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ac5ef2-1a2e-4961-a823-46f22a2636e0 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:36.218326 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d9e92f-6e58-4048-b10f-cf3cb53d56c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:36.219039 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d9e92f-6e58-4048-b10f-cf3cb53d56c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:36.219986 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d9e92f-6e58-4048-b10f-cf3cb53d56c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:36.219986 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d9e92f-6e58-4048-b10f-cf3cb53d56c2 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:36.220419 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d9e92f-6e58-4048-b10f-cf3cb53d56c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:36.222332 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d9e92f-6e58-4048-b10f-cf3cb53d56c2 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:36.302578 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1153/2311] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 17 19:31:36 2022] POST /identity/v3/users => generated 287 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:31:36.305658 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a8d04b-3689-4edf-bcc9-dcf07b505262 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:36.317285 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a8d04b-3689-4edf-bcc9-dcf07b505262 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:36.323133 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e874cc4a-9b0c-4030-a72f-887208c0eddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:36.323725 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e874cc4a-9b0c-4030-a72f-887208c0eddc admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:36.324616 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e874cc4a-9b0c-4030-a72f-887208c0eddc admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:36.325076 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e874cc4a-9b0c-4030-a72f-887208c0eddc admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:36.325868 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e874cc4a-9b0c-4030-a72f-887208c0eddc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:36.327536 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e874cc4a-9b0c-4030-a72f-887208c0eddc admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:36.338711 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1159/2312] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:31:36 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:31:36.342706 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d9e92f-6e58-4048-b10f-cf3cb53d56c2 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:36.358899 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d9e92f-6e58-4048-b10f-cf3cb53d56c2 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:36.364773 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2219e24-44c2-4048-bded-844fa0c9051f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:36.365451 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2219e24-44c2-4048-bded-844fa0c9051f admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:36.365726 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2219e24-44c2-4048-bded-844fa0c9051f admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:36.366003 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2219e24-44c2-4048-bded-844fa0c9051f admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:36.367332 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2219e24-44c2-4048-bded-844fa0c9051f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:36.369562 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2219e24-44c2-4048-bded-844fa0c9051f admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:36.389896 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1154/2313] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:31:36 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:31:36.393774 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e874cc4a-9b0c-4030-a72f-887208c0eddc admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:36.406412 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e874cc4a-9b0c-4030-a72f-887208c0eddc admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:36.415966 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb40952-10e0-4ef3-a795-ddf70be5834f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:36.416906 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb40952-10e0-4ef3-a795-ddf70be5834f admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:36.417182 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb40952-10e0-4ef3-a795-ddf70be5834f admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:36.419368 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb40952-10e0-4ef3-a795-ddf70be5834f admin admin] PATH_INFO: `/v3/projects/58d67fd0a380440ab2a5153f8535781f/users/892dfab202ad4a84a5c5e1d5b120aa28/roles/114e6a16e63447d1924707ace5aace25` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:36.441636 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb40952-10e0-4ef3-a795-ddf70be5834f admin admin] RBAC: Authorizing `identity:create_grant(project_id=58d67fd0a380440ab2a5153f8535781f, user_id=892dfab202ad4a84a5c5e1d5b120aa28, role_id=114e6a16e63447d1924707ace5aace25)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:36.444489 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb40952-10e0-4ef3-a795-ddf70be5834f admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:36.553433 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1160/2314] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:31:36 2022] PUT /identity/v3/projects/58d67fd0a380440ab2a5153f8535781f/users/892dfab202ad4a84a5c5e1d5b120aa28/roles/114e6a16e63447d1924707ace5aace25 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:31:36.562544 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082f065b-28d6-42fe-ab52-d7ef60e54e0c None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:36.562835 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082f065b-28d6-42fe-ab52-d7ef60e54e0c None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:36.563106 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082f065b-28d6-42fe-ab52-d7ef60e54e0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:36.592552 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-082f065b-28d6-42fe-ab52-d7ef60e54e0c None None] MFA Rules not processed for user `892dfab202ad4a84a5c5e1d5b120aa28`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:31:36.593846 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-082f065b-28d6-42fe-ab52-d7ef60e54e0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:36.667620 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1155/2315] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:31:36 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:31:36.671783 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb40952-10e0-4ef3-a795-ddf70be5834f admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:36.756746 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb40952-10e0-4ef3-a795-ddf70be5834f admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:36.764959 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0398d7d0-5873-4909-8cac-2aa9e24e8da5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:36.764959 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0398d7d0-5873-4909-8cac-2aa9e24e8da5 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:36.764959 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0398d7d0-5873-4909-8cac-2aa9e24e8da5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:36.764959 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0398d7d0-5873-4909-8cac-2aa9e24e8da5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:36.765258 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0398d7d0-5873-4909-8cac-2aa9e24e8da5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:36.765791 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0398d7d0-5873-4909-8cac-2aa9e24e8da5 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:36.881437 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1161/2316] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 17 19:31:36 2022] POST /identity/v3/projects => generated 375 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:31:36.882703 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082f065b-28d6-42fe-ab52-d7ef60e54e0c None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:36.899841 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082f065b-28d6-42fe-ab52-d7ef60e54e0c None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:36.984282 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8d86a5-1b1c-4b6b-a88c-aad95c6ea503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:36.985089 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8d86a5-1b1c-4b6b-a88c-aad95c6ea503 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:36.986255 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8d86a5-1b1c-4b6b-a88c-aad95c6ea503 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:36.988234 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8d86a5-1b1c-4b6b-a88c-aad95c6ea503 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:36.990052 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8d86a5-1b1c-4b6b-a88c-aad95c6ea503 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:37.002263 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8d86a5-1b1c-4b6b-a88c-aad95c6ea503 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:37.111509 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1156/2317] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 17 19:31:36 2022] POST /identity/v3/users => generated 294 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:31:37.112745 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0398d7d0-5873-4909-8cac-2aa9e24e8da5 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:37.122563 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0398d7d0-5873-4909-8cac-2aa9e24e8da5 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:37.127633 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0344cdc-c038-4195-98c7-bc74410fd978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:37.128340 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0344cdc-c038-4195-98c7-bc74410fd978 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:37.128668 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0344cdc-c038-4195-98c7-bc74410fd978 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:37.128944 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0344cdc-c038-4195-98c7-bc74410fd978 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:37.129679 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0344cdc-c038-4195-98c7-bc74410fd978 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:37.131320 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0344cdc-c038-4195-98c7-bc74410fd978 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:37.142304 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1162/2318] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:31:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:31:37.146543 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8d86a5-1b1c-4b6b-a88c-aad95c6ea503 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:37.157178 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8d86a5-1b1c-4b6b-a88c-aad95c6ea503 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:37.162564 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5287bcd2-f28c-4ba5-89c2-6deeae00e4cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:37.163080 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5287bcd2-f28c-4ba5-89c2-6deeae00e4cf admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:37.163380 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5287bcd2-f28c-4ba5-89c2-6deeae00e4cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:37.163679 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5287bcd2-f28c-4ba5-89c2-6deeae00e4cf admin admin] PATH_INFO: `/v3/projects/6e7d800b324046d3914f85cc72c90fcf/users/08b506f6a59e4b428a4b307472e80eb2/roles/60af5088793e4379b88821a86552a344` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:37.178673 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5287bcd2-f28c-4ba5-89c2-6deeae00e4cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e7d800b324046d3914f85cc72c90fcf, user_id=08b506f6a59e4b428a4b307472e80eb2, role_id=60af5088793e4379b88821a86552a344)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:37.181419 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5287bcd2-f28c-4ba5-89c2-6deeae00e4cf admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:37.325320 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1157/2319] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:31:37 2022] PUT /identity/v3/projects/6e7d800b324046d3914f85cc72c90fcf/users/08b506f6a59e4b428a4b307472e80eb2/roles/60af5088793e4379b88821a86552a344 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:31:37.329384 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0344cdc-c038-4195-98c7-bc74410fd978 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:37.339063 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0344cdc-c038-4195-98c7-bc74410fd978 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:37.400284 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d75cdd-15dc-43e2-a53f-bd222f1f97a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:37.401079 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d75cdd-15dc-43e2-a53f-bd222f1f97a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:37.401353 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d75cdd-15dc-43e2-a53f-bd222f1f97a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:37.401624 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d75cdd-15dc-43e2-a53f-bd222f1f97a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:37.402839 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d75cdd-15dc-43e2-a53f-bd222f1f97a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:37.404461 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d75cdd-15dc-43e2-a53f-bd222f1f97a7 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:37.414752 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1163/2320] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:31:37 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:31:37.419134 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5287bcd2-f28c-4ba5-89c2-6deeae00e4cf admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:37.429164 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5287bcd2-f28c-4ba5-89c2-6deeae00e4cf admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:37.438760 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4013062-7041-4e79-b3ea-92b326b2d967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:37.438760 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4013062-7041-4e79-b3ea-92b326b2d967 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:37.438760 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4013062-7041-4e79-b3ea-92b326b2d967 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:37.438760 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4013062-7041-4e79-b3ea-92b326b2d967 admin admin] PATH_INFO: `/v3/projects/6e7d800b324046d3914f85cc72c90fcf/users/08b506f6a59e4b428a4b307472e80eb2/roles/60af5088793e4379b88821a86552a344` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:37.439334 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4013062-7041-4e79-b3ea-92b326b2d967 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e7d800b324046d3914f85cc72c90fcf, user_id=08b506f6a59e4b428a4b307472e80eb2, role_id=60af5088793e4379b88821a86552a344)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:37.442098 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4013062-7041-4e79-b3ea-92b326b2d967 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:37.454057 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1158/2321] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:31:37 2022] PUT /identity/v3/projects/6e7d800b324046d3914f85cc72c90fcf/users/08b506f6a59e4b428a4b307472e80eb2/roles/60af5088793e4379b88821a86552a344 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:31:37.465471 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b20cf4-b64a-4839-994f-244cb57f30c6 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:37.469813 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b20cf4-b64a-4839-994f-244cb57f30c6 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:37.469813 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b20cf4-b64a-4839-994f-244cb57f30c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:37.493648 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-c5b20cf4-b64a-4839-994f-244cb57f30c6 None None] MFA Rules not processed for user `08b506f6a59e4b428a4b307472e80eb2`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:31:37.494132 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-c5b20cf4-b64a-4839-994f-244cb57f30c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:37.560304 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1164/2322] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:31:37 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:31:37.571249 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4013062-7041-4e79-b3ea-92b326b2d967 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:37.575660 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-f4013062-7041-4e79-b3ea-92b326b2d967 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:37.631670 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4013062-7041-4e79-b3ea-92b326b2d967 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:37.635303 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7121b7-44ad-4cdb-80e5-751d93f8c8be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd05409b38ae346458abfe41ebda1c59e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:37.636139 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7121b7-44ad-4cdb-80e5-751d93f8c8be service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:37.636932 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7121b7-44ad-4cdb-80e5-751d93f8c8be service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:37.637797 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7121b7-44ad-4cdb-80e5-751d93f8c8be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:37.697551 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7121b7-44ad-4cdb-80e5-751d93f8c8be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:37.699589 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7121b7-44ad-4cdb-80e5-751d93f8c8be service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:37.708457 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1159/2323] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:31:37 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:31:43.231354 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b20cf4-b64a-4839-994f-244cb57f30c6 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:43.311474 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b20cf4-b64a-4839-994f-244cb57f30c6 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:43.325015 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fc3679-63fa-439f-8c38-6ef47705c5a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:43.325982 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fc3679-63fa-439f-8c38-6ef47705c5a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:43.326264 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fc3679-63fa-439f-8c38-6ef47705c5a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:43.326756 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fc3679-63fa-439f-8c38-6ef47705c5a3 admin admin] PATH_INFO: `/v3/users/892dfab202ad4a84a5c5e1d5b120aa28` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:43.328752 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fc3679-63fa-439f-8c38-6ef47705c5a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=892dfab202ad4a84a5c5e1d5b120aa28)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:43.330776 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fc3679-63fa-439f-8c38-6ef47705c5a3 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:43.346511 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7121b7-44ad-4cdb-80e5-751d93f8c8be service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:43.356236 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-4d7121b7-44ad-4cdb-80e5-751d93f8c8be service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:43.363792 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7121b7-44ad-4cdb-80e5-751d93f8c8be service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:43.367510 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15499ed4-be11-4e06-90bb-7e1d1e87ec26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:43.368552 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15499ed4-be11-4e06-90bb-7e1d1e87ec26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:43.368836 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15499ed4-be11-4e06-90bb-7e1d1e87ec26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:43.369084 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15499ed4-be11-4e06-90bb-7e1d1e87ec26 admin admin] PATH_INFO: `/v3/users/11afb29192e04ac3added23ac5cae56e` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:43.370251 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15499ed4-be11-4e06-90bb-7e1d1e87ec26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=11afb29192e04ac3added23ac5cae56e)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:43.372783 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15499ed4-be11-4e06-90bb-7e1d1e87ec26 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:43.578676 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-99fc3679-63fa-439f-8c38-6ef47705c5a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '892dfab202ad4a84a5c5e1d5b120aa28'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:43.581455 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-15499ed4-be11-4e06-90bb-7e1d1e87ec26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11afb29192e04ac3added23ac5cae56e'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:43.582559 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-15499ed4-be11-4e06-90bb-7e1d1e87ec26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11afb29192e04ac3added23ac5cae56e'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:43.586307 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-99fc3679-63fa-439f-8c38-6ef47705c5a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '892dfab202ad4a84a5c5e1d5b120aa28'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:43.601886 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-99fc3679-63fa-439f-8c38-6ef47705c5a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '892dfab202ad4a84a5c5e1d5b120aa28'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:43.602693 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-99fc3679-63fa-439f-8c38-6ef47705c5a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '892dfab202ad4a84a5c5e1d5b120aa28'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:43.607714 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-99fc3679-63fa-439f-8c38-6ef47705c5a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '892dfab202ad4a84a5c5e1d5b120aa28'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:43.607984 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-15499ed4-be11-4e06-90bb-7e1d1e87ec26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11afb29192e04ac3added23ac5cae56e'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:43.700616 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1165/2324] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 17 19:31:43 2022] DELETE /identity/v3/users/892dfab202ad4a84a5c5e1d5b120aa28 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:31:43.705098 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee821ff-4f36-42f9-9751-2c08033a9b6f None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:43.705365 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee821ff-4f36-42f9-9751-2c08033a9b6f None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:43.705640 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee821ff-4f36-42f9-9751-2c08033a9b6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:43.722543 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-7ee821ff-4f36-42f9-9751-2c08033a9b6f None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:31:43.724178 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-7ee821ff-4f36-42f9-9751-2c08033a9b6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:43.776719 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-15499ed4-be11-4e06-90bb-7e1d1e87ec26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11afb29192e04ac3added23ac5cae56e'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:43.786745 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-15499ed4-be11-4e06-90bb-7e1d1e87ec26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11afb29192e04ac3added23ac5cae56e'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:43.789237 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1166/2325] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:31:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 17 19:31:43.799413 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee821ff-4f36-42f9-9751-2c08033a9b6f None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:43.802478 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-7ee821ff-4f36-42f9-9751-2c08033a9b6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:43.815952 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1160/2326] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 17 19:31:43 2022] DELETE /identity/v3/users/11afb29192e04ac3added23ac5cae56e => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 17 19:31:43.817156 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb6220c-6995-42c5-87c3-f3d93819f866 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:43.817452 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb6220c-6995-42c5-87c3-f3d93819f866 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:43.817734 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb6220c-6995-42c5-87c3-f3d93819f866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:43.839864 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-0eb6220c-6995-42c5-87c3-f3d93819f866 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:31:43.841192 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-0eb6220c-6995-42c5-87c3-f3d93819f866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:43.862587 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee821ff-4f36-42f9-9751-2c08033a9b6f None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:43.916691 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1161/2327] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:31:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:31:43.923808 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761c02ac-22c6-4057-a95b-d2b26130b309 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:43.927206 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb6220c-6995-42c5-87c3-f3d93819f866 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:43.928598 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-0eb6220c-6995-42c5-87c3-f3d93819f866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:43.931660 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761c02ac-22c6-4057-a95b-d2b26130b309 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:43.932029 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761c02ac-22c6-4057-a95b-d2b26130b309 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:43.932321 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761c02ac-22c6-4057-a95b-d2b26130b309 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:43.943247 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb6220c-6995-42c5-87c3-f3d93819f866 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:43.947662 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42061bce-b78a-4630-afbb-676b4e2fc9cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:43.948378 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42061bce-b78a-4630-afbb-676b4e2fc9cd service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:43.949226 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42061bce-b78a-4630-afbb-676b4e2fc9cd service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:43.949474 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42061bce-b78a-4630-afbb-676b4e2fc9cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:43.959360 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761c02ac-22c6-4057-a95b-d2b26130b309 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:43.960096 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42061bce-b78a-4630-afbb-676b4e2fc9cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:43.961674 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42061bce-b78a-4630-afbb-676b4e2fc9cd service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:43.963663 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761c02ac-22c6-4057-a95b-d2b26130b309 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:43.975428 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1162/2328] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:31:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 17 19:31:43.985442 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1167/2329] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:31:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 17 19:31:44.354455 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42061bce-b78a-4630-afbb-676b4e2fc9cd service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:44.356456 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-42061bce-b78a-4630-afbb-676b4e2fc9cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:44.379776 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42061bce-b78a-4630-afbb-676b4e2fc9cd service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:44.391271 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff1894c-12b7-4dba-8ea9-40febad7bb91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:44.392184 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff1894c-12b7-4dba-8ea9-40febad7bb91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:44.392465 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff1894c-12b7-4dba-8ea9-40febad7bb91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:44.392738 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff1894c-12b7-4dba-8ea9-40febad7bb91 admin admin] PATH_INFO: `/v3/users/41831da07b194c6a930a2a3377a7855c` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:44.397906 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff1894c-12b7-4dba-8ea9-40febad7bb91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41831da07b194c6a930a2a3377a7855c)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:44.400168 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff1894c-12b7-4dba-8ea9-40febad7bb91 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:44.525742 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761c02ac-22c6-4057-a95b-d2b26130b309 service neutron] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:44.530326 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-761c02ac-22c6-4057-a95b-d2b26130b309 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:44.545420 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761c02ac-22c6-4057-a95b-d2b26130b309 service neutron] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:44.552646 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16069e76-baef-4887-a825-9fdb2d7de015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:44.553579 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16069e76-baef-4887-a825-9fdb2d7de015 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:44.553856 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16069e76-baef-4887-a825-9fdb2d7de015 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:44.554247 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16069e76-baef-4887-a825-9fdb2d7de015 admin admin] PATH_INFO: `/v3/projects/37cad1bf9c954fd3b4736d5823dce622` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:44.555827 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16069e76-baef-4887-a825-9fdb2d7de015 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37cad1bf9c954fd3b4736d5823dce622)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:44.559647 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16069e76-baef-4887-a825-9fdb2d7de015 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:44.643031 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-cff1894c-12b7-4dba-8ea9-40febad7bb91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41831da07b194c6a930a2a3377a7855c'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:44.645399 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-cff1894c-12b7-4dba-8ea9-40febad7bb91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41831da07b194c6a930a2a3377a7855c'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:44.658668 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-cff1894c-12b7-4dba-8ea9-40febad7bb91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41831da07b194c6a930a2a3377a7855c'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:44.702981 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-16069e76-baef-4887-a825-9fdb2d7de015 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37cad1bf9c954fd3b4736d5823dce622', 'request_id': 'req-16069e76-baef-4887-a825-9fdb2d7de015'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:44.710095 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-16069e76-baef-4887-a825-9fdb2d7de015 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37cad1bf9c954fd3b4736d5823dce622', 'request_id': 'req-16069e76-baef-4887-a825-9fdb2d7de015'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:44.716991 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-16069e76-baef-4887-a825-9fdb2d7de015 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37cad1bf9c954fd3b4736d5823dce622', 'request_id': 'req-16069e76-baef-4887-a825-9fdb2d7de015'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:44.730779 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-cff1894c-12b7-4dba-8ea9-40febad7bb91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41831da07b194c6a930a2a3377a7855c'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:44.738305 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-cff1894c-12b7-4dba-8ea9-40febad7bb91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41831da07b194c6a930a2a3377a7855c'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:44.744579 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1163/2330] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 17 19:31:44 2022] DELETE /identity/v3/users/41831da07b194c6a930a2a3377a7855c => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:31:44.745380 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff1894c-12b7-4dba-8ea9-40febad7bb91 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:44.747507 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-cff1894c-12b7-4dba-8ea9-40febad7bb91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:44.817545 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-16069e76-baef-4887-a825-9fdb2d7de015 admin admin] The token cache is being invalidate because project 37cad1bf9c954fd3b4736d5823dce622 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 17 19:31:44.818587 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-16069e76-baef-4887-a825-9fdb2d7de015 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:44.819758 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-16069e76-baef-4887-a825-9fdb2d7de015 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:44.822151 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1168/2331] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 17 19:31:44 2022] DELETE /identity/v3/projects/37cad1bf9c954fd3b4736d5823dce622 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:31:44.824456 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aee256-50eb-4b38-b475-daa788a1360b None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:44.824775 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aee256-50eb-4b38-b475-daa788a1360b None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:44.825150 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aee256-50eb-4b38-b475-daa788a1360b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:44.837542 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff1894c-12b7-4dba-8ea9-40febad7bb91 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:44.842728 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4a79f2-af33-4c4d-8fa0-6a4a127b855c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:44.843663 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4a79f2-af33-4c4d-8fa0-6a4a127b855c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:44.843947 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4a79f2-af33-4c4d-8fa0-6a4a127b855c admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:44.844224 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4a79f2-af33-4c4d-8fa0-6a4a127b855c admin admin] PATH_INFO: `/v3/projects/58d67fd0a380440ab2a5153f8535781f` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:44.845488 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4a79f2-af33-4c4d-8fa0-6a4a127b855c admin admin] RBAC: Authorizing `identity:delete_project(project_id=58d67fd0a380440ab2a5153f8535781f)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:44.849792 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-d2aee256-50eb-4b38-b475-daa788a1360b None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:31:44.851117 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-d2aee256-50eb-4b38-b475-daa788a1360b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:44.856158 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4a79f2-af33-4c4d-8fa0-6a4a127b855c admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:44.863165 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1169/2332] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:31:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:31:44.863165 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aee256-50eb-4b38-b475-daa788a1360b None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:44.863165 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-d2aee256-50eb-4b38-b475-daa788a1360b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:44.884602 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aee256-50eb-4b38-b475-daa788a1360b None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:44.900284 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ff0e94-60d8-43ce-b8ab-94395975a72f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:44.901582 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ff0e94-60d8-43ce-b8ab-94395975a72f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:44.902663 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ff0e94-60d8-43ce-b8ab-94395975a72f admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:44.903137 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ff0e94-60d8-43ce-b8ab-94395975a72f admin admin] PATH_INFO: `/v3/users/63d29b10bc7f4cde9c0ad3523b0380e9` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:44.905074 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ff0e94-60d8-43ce-b8ab-94395975a72f admin admin] RBAC: Authorizing `identity:delete_user(user_id=63d29b10bc7f4cde9c0ad3523b0380e9)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:44.908028 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ff0e94-60d8-43ce-b8ab-94395975a72f admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:44.973142 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-3d4a79f2-af33-4c4d-8fa0-6a4a127b855c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58d67fd0a380440ab2a5153f8535781f', 'request_id': 'req-3d4a79f2-af33-4c4d-8fa0-6a4a127b855c'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:44.979766 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-3d4a79f2-af33-4c4d-8fa0-6a4a127b855c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58d67fd0a380440ab2a5153f8535781f', 'request_id': 'req-3d4a79f2-af33-4c4d-8fa0-6a4a127b855c'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:45.124872 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-3d4a79f2-af33-4c4d-8fa0-6a4a127b855c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58d67fd0a380440ab2a5153f8535781f', 'request_id': 'req-3d4a79f2-af33-4c4d-8fa0-6a4a127b855c'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:45.136193 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-3d4a79f2-af33-4c4d-8fa0-6a4a127b855c admin admin] The token cache is being invalidate because project 58d67fd0a380440ab2a5153f8535781f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 17 19:31:45.136477 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-3d4a79f2-af33-4c4d-8fa0-6a4a127b855c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:45.137114 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-3d4a79f2-af33-4c4d-8fa0-6a4a127b855c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:45.138813 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1164/2333] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 17 19:31:44 2022] DELETE /identity/v3/projects/58d67fd0a380440ab2a5153f8535781f => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:31:45.141312 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4a79f2-af33-4c4d-8fa0-6a4a127b855c admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:45.142829 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-3d4a79f2-af33-4c4d-8fa0-6a4a127b855c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:45.164705 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-a9ff0e94-60d8-43ce-b8ab-94395975a72f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63d29b10bc7f4cde9c0ad3523b0380e9'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:45.172630 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-a9ff0e94-60d8-43ce-b8ab-94395975a72f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63d29b10bc7f4cde9c0ad3523b0380e9'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:45.191052 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-a9ff0e94-60d8-43ce-b8ab-94395975a72f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63d29b10bc7f4cde9c0ad3523b0380e9'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:45.191802 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-a9ff0e94-60d8-43ce-b8ab-94395975a72f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63d29b10bc7f4cde9c0ad3523b0380e9'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:45.192517 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-a9ff0e94-60d8-43ce-b8ab-94395975a72f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63d29b10bc7f4cde9c0ad3523b0380e9'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:45.211706 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4a79f2-af33-4c4d-8fa0-6a4a127b855c admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:45.216375 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcd2f8d-8a29-44fa-9b6b-e24571d467f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:45.217120 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcd2f8d-8a29-44fa-9b6b-e24571d467f6 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:45.217405 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcd2f8d-8a29-44fa-9b6b-e24571d467f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:45.217705 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcd2f8d-8a29-44fa-9b6b-e24571d467f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:45.219649 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-8dcd2f8d-8a29-44fa-9b6b-e24571d467f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:45.283371 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1170/2334] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 17 19:31:44 2022] DELETE /identity/v3/users/63d29b10bc7f4cde9c0ad3523b0380e9 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:31:45.284276 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ff0e94-60d8-43ce-b8ab-94395975a72f admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:45.285622 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-a9ff0e94-60d8-43ce-b8ab-94395975a72f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:45.286962 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcd2f8d-8a29-44fa-9b6b-e24571d467f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:45.288557 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcd2f8d-8a29-44fa-9b6b-e24571d467f6 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:45.302753 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1165/2335] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:31:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:31:45.354489 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ff0e94-60d8-43ce-b8ab-94395975a72f admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:45.360204 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b31a3b4-773a-43e1-ba39-b6c9fc9e2375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:45.361264 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b31a3b4-773a-43e1-ba39-b6c9fc9e2375 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:45.361544 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b31a3b4-773a-43e1-ba39-b6c9fc9e2375 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:45.361965 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b31a3b4-773a-43e1-ba39-b6c9fc9e2375 admin admin] PATH_INFO: `/v3/users/08b506f6a59e4b428a4b307472e80eb2` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:45.363554 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b31a3b4-773a-43e1-ba39-b6c9fc9e2375 admin admin] RBAC: Authorizing `identity:delete_user(user_id=08b506f6a59e4b428a4b307472e80eb2)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:45.365791 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b31a3b4-773a-43e1-ba39-b6c9fc9e2375 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:45.564858 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-4b31a3b4-773a-43e1-ba39-b6c9fc9e2375 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08b506f6a59e4b428a4b307472e80eb2'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:45.575709 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-4b31a3b4-773a-43e1-ba39-b6c9fc9e2375 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08b506f6a59e4b428a4b307472e80eb2'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:45.590152 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-4b31a3b4-773a-43e1-ba39-b6c9fc9e2375 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08b506f6a59e4b428a4b307472e80eb2'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:45.591048 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-4b31a3b4-773a-43e1-ba39-b6c9fc9e2375 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08b506f6a59e4b428a4b307472e80eb2'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:45.592291 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-4b31a3b4-773a-43e1-ba39-b6c9fc9e2375 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08b506f6a59e4b428a4b307472e80eb2'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:45.681577 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1171/2336] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 17 19:31:45 2022] DELETE /identity/v3/users/08b506f6a59e4b428a4b307472e80eb2 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:31:45.815888 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcd2f8d-8a29-44fa-9b6b-e24571d467f6 service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:45.817298 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-8dcd2f8d-8a29-44fa-9b6b-e24571d467f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:45.830114 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcd2f8d-8a29-44fa-9b6b-e24571d467f6 service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:45.881497 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b31a3b4-773a-43e1-ba39-b6c9fc9e2375 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:45.882565 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-4b31a3b4-773a-43e1-ba39-b6c9fc9e2375 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:45.893673 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f833e8a-63d4-4981-b6e6-da753be418b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:45.894518 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f833e8a-63d4-4981-b6e6-da753be418b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:45.894794 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f833e8a-63d4-4981-b6e6-da753be418b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:45.895975 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f833e8a-63d4-4981-b6e6-da753be418b1 admin admin] PATH_INFO: `/v3/projects/fa950f327a934d89a2b848ceedd403e7` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:45.897437 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f833e8a-63d4-4981-b6e6-da753be418b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa950f327a934d89a2b848ceedd403e7)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:45.906661 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f833e8a-63d4-4981-b6e6-da753be418b1 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:45.998148 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b31a3b4-773a-43e1-ba39-b6c9fc9e2375 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:46.002876 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb788162-bde0-4b17-9d5f-abc3627f9a05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:46.003871 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb788162-bde0-4b17-9d5f-abc3627f9a05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:46.004153 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb788162-bde0-4b17-9d5f-abc3627f9a05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:46.004428 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb788162-bde0-4b17-9d5f-abc3627f9a05 admin admin] PATH_INFO: `/v3/projects/d877e10950e8462fbf52da32de0a502a` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:46.005752 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb788162-bde0-4b17-9d5f-abc3627f9a05 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d877e10950e8462fbf52da32de0a502a)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:46.008056 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb788162-bde0-4b17-9d5f-abc3627f9a05 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:46.043584 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0f833e8a-63d4-4981-b6e6-da753be418b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa950f327a934d89a2b848ceedd403e7', 'request_id': 'req-0f833e8a-63d4-4981-b6e6-da753be418b1'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:46.049905 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0f833e8a-63d4-4981-b6e6-da753be418b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa950f327a934d89a2b848ceedd403e7', 'request_id': 'req-0f833e8a-63d4-4981-b6e6-da753be418b1'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:46.193587 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-eb788162-bde0-4b17-9d5f-abc3627f9a05 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd877e10950e8462fbf52da32de0a502a', 'request_id': 'req-eb788162-bde0-4b17-9d5f-abc3627f9a05'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:46.208272 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-eb788162-bde0-4b17-9d5f-abc3627f9a05 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd877e10950e8462fbf52da32de0a502a', 'request_id': 'req-eb788162-bde0-4b17-9d5f-abc3627f9a05'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:46.210078 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0f833e8a-63d4-4981-b6e6-da753be418b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa950f327a934d89a2b848ceedd403e7', 'request_id': 'req-0f833e8a-63d4-4981-b6e6-da753be418b1'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:46.232149 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-eb788162-bde0-4b17-9d5f-abc3627f9a05 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd877e10950e8462fbf52da32de0a502a', 'request_id': 'req-eb788162-bde0-4b17-9d5f-abc3627f9a05'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:46.235513 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0f833e8a-63d4-4981-b6e6-da753be418b1 admin admin] The token cache is being invalidate because project fa950f327a934d89a2b848ceedd403e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 17 19:31:46.236409 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0f833e8a-63d4-4981-b6e6-da753be418b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:46.237292 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0f833e8a-63d4-4981-b6e6-da753be418b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:46.240085 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1166/2337] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 17 19:31:45 2022] DELETE /identity/v3/projects/fa950f327a934d89a2b848ceedd403e7 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:31:46.246516 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a1d001-eb47-45ac-bdbd-23228012d62e None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:46.246516 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a1d001-eb47-45ac-bdbd-23228012d62e None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:46.246881 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a1d001-eb47-45ac-bdbd-23228012d62e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:46.271009 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-b7a1d001-eb47-45ac-bdbd-23228012d62e None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:31:46.272233 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b7a1d001-eb47-45ac-bdbd-23228012d62e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:46.316551 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-eb788162-bde0-4b17-9d5f-abc3627f9a05 admin admin] The token cache is being invalidate because project d877e10950e8462fbf52da32de0a502a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 17 19:31:46.316995 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-eb788162-bde0-4b17-9d5f-abc3627f9a05 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:46.317700 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-eb788162-bde0-4b17-9d5f-abc3627f9a05 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:46.319958 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1172/2338] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 17 19:31:45 2022] DELETE /identity/v3/projects/d877e10950e8462fbf52da32de0a502a => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:31:46.324541 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb788162-bde0-4b17-9d5f-abc3627f9a05 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:46.326099 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-eb788162-bde0-4b17-9d5f-abc3627f9a05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:46.364380 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1167/2339] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:31:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 17 19:31:46.373581 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b66eb81-ced1-46bf-a530-be2d9c51ca9a None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:46.373856 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b66eb81-ced1-46bf-a530-be2d9c51ca9a None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:46.374137 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b66eb81-ced1-46bf-a530-be2d9c51ca9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:46.419819 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-8b66eb81-ced1-46bf-a530-be2d9c51ca9a None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:31:46.421400 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-8b66eb81-ced1-46bf-a530-be2d9c51ca9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:46.443774 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1168/2340] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:31:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:31:46.444739 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b66eb81-ced1-46bf-a530-be2d9c51ca9a None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:46.447808 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-8b66eb81-ced1-46bf-a530-be2d9c51ca9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:46.455547 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb788162-bde0-4b17-9d5f-abc3627f9a05 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:46.462546 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd9311b-3cf3-4f3b-9f32-2db2ea1adb61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:46.462698 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b66eb81-ced1-46bf-a530-be2d9c51ca9a None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:46.463587 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd9311b-3cf3-4f3b-9f32-2db2ea1adb61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:46.463865 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd9311b-3cf3-4f3b-9f32-2db2ea1adb61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:46.464158 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd9311b-3cf3-4f3b-9f32-2db2ea1adb61 admin admin] PATH_INFO: `/v3/users/3333e72e51874b6fa9bb9d673d0a4a34` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:46.465630 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd9311b-3cf3-4f3b-9f32-2db2ea1adb61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3333e72e51874b6fa9bb9d673d0a4a34)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:46.468309 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d275b9d9-529d-40d5-9214-7d25ef9d8f4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:46.468584 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd9311b-3cf3-4f3b-9f32-2db2ea1adb61 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:46.471052 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d275b9d9-529d-40d5-9214-7d25ef9d8f4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:46.471052 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d275b9d9-529d-40d5-9214-7d25ef9d8f4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:46.471052 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d275b9d9-529d-40d5-9214-7d25ef9d8f4d admin admin] PATH_INFO: `/v3/projects/6e7d800b324046d3914f85cc72c90fcf` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:46.471801 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d275b9d9-529d-40d5-9214-7d25ef9d8f4d admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e7d800b324046d3914f85cc72c90fcf)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:46.474254 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d275b9d9-529d-40d5-9214-7d25ef9d8f4d admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:46.684733 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-cfd9311b-3cf3-4f3b-9f32-2db2ea1adb61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3333e72e51874b6fa9bb9d673d0a4a34'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:46.687975 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d275b9d9-529d-40d5-9214-7d25ef9d8f4d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e7d800b324046d3914f85cc72c90fcf', 'request_id': 'req-d275b9d9-529d-40d5-9214-7d25ef9d8f4d'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:46.693748 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-cfd9311b-3cf3-4f3b-9f32-2db2ea1adb61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3333e72e51874b6fa9bb9d673d0a4a34'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:46.695170 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d275b9d9-529d-40d5-9214-7d25ef9d8f4d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e7d800b324046d3914f85cc72c90fcf', 'request_id': 'req-d275b9d9-529d-40d5-9214-7d25ef9d8f4d'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:46.706458 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-cfd9311b-3cf3-4f3b-9f32-2db2ea1adb61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3333e72e51874b6fa9bb9d673d0a4a34'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:46.707267 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-cfd9311b-3cf3-4f3b-9f32-2db2ea1adb61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3333e72e51874b6fa9bb9d673d0a4a34'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:46.707994 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-cfd9311b-3cf3-4f3b-9f32-2db2ea1adb61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3333e72e51874b6fa9bb9d673d0a4a34'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:46.779292 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d275b9d9-529d-40d5-9214-7d25ef9d8f4d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e7d800b324046d3914f85cc72c90fcf', 'request_id': 'req-d275b9d9-529d-40d5-9214-7d25ef9d8f4d'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:46.792901 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d275b9d9-529d-40d5-9214-7d25ef9d8f4d admin admin] The token cache is being invalidate because project 6e7d800b324046d3914f85cc72c90fcf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 17 19:31:46.793188 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d275b9d9-529d-40d5-9214-7d25ef9d8f4d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:46.793860 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d275b9d9-529d-40d5-9214-7d25ef9d8f4d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:46.795479 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1169/2341] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 17 19:31:46 2022] DELETE /identity/v3/projects/6e7d800b324046d3914f85cc72c90fcf => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:31:46.796240 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d275b9d9-529d-40d5-9214-7d25ef9d8f4d admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:46.797519 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d275b9d9-529d-40d5-9214-7d25ef9d8f4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:46.859540 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1173/2342] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 17 19:31:46 2022] DELETE /identity/v3/users/3333e72e51874b6fa9bb9d673d0a4a34 => generated 0 bytes in 536 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:31:46.861079 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547edb18-4ff4-420e-83c4-37aa921c0cbc None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:46.861344 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547edb18-4ff4-420e-83c4-37aa921c0cbc None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:46.861678 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547edb18-4ff4-420e-83c4-37aa921c0cbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:46.864300 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d275b9d9-529d-40d5-9214-7d25ef9d8f4d admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:46.894431 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-547edb18-4ff4-420e-83c4-37aa921c0cbc None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:31:46.896222 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-547edb18-4ff4-420e-83c4-37aa921c0cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:46.923418 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f36adba-7981-492a-998e-e501c00d3a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:46.924110 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f36adba-7981-492a-998e-e501c00d3a41 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:46.924491 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f36adba-7981-492a-998e-e501c00d3a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:46.924658 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f36adba-7981-492a-998e-e501c00d3a41 admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:46.925381 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f36adba-7981-492a-998e-e501c00d3a41 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:31:46.925688 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f36adba-7981-492a-998e-e501c00d3a41 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:46.927305 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f36adba-7981-492a-998e-e501c00d3a41 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:46.947522 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1170/2343] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 17 19:31:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:31:46.947522 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f36adba-7981-492a-998e-e501c00d3a41 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:46.959932 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f36adba-7981-492a-998e-e501c00d3a41 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:46.963557 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1174/2344] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:31:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:31:46.965532 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467ca133-8575-4a17-96dd-e78ba2b346b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:46.966191 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467ca133-8575-4a17-96dd-e78ba2b346b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:46.966471 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467ca133-8575-4a17-96dd-e78ba2b346b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:46.966741 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467ca133-8575-4a17-96dd-e78ba2b346b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:46.967442 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467ca133-8575-4a17-96dd-e78ba2b346b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:46.969831 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467ca133-8575-4a17-96dd-e78ba2b346b4 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:46.975454 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f747d27-6803-467b-b207-95b5914d68cb None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:46.975454 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f747d27-6803-467b-b207-95b5914d68cb None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:46.975454 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f747d27-6803-467b-b207-95b5914d68cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:46.999344 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-2f747d27-6803-467b-b207-95b5914d68cb None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:31:47.000870 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-2f747d27-6803-467b-b207-95b5914d68cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:47.022987 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1175/2345] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:31:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 17 19:31:47.023622 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f747d27-6803-467b-b207-95b5914d68cb None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:47.034334 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f747d27-6803-467b-b207-95b5914d68cb None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:47.039340 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05da7d7b-d940-4ad6-8650-091b913e54af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:47.040049 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05da7d7b-d940-4ad6-8650-091b913e54af admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:47.040335 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05da7d7b-d940-4ad6-8650-091b913e54af admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:47.040640 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05da7d7b-d940-4ad6-8650-091b913e54af admin admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:47.041410 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05da7d7b-d940-4ad6-8650-091b913e54af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:31:47.041715 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05da7d7b-d940-4ad6-8650-091b913e54af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:47.045562 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05da7d7b-d940-4ad6-8650-091b913e54af admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:47.059837 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1176/2346] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 17 19:31:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:31:47.064222 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05da7d7b-d940-4ad6-8650-091b913e54af admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:47.078570 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05da7d7b-d940-4ad6-8650-091b913e54af admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:47.087622 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae6eea5-1941-43a4-84ca-a58e99c17a46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:47.088417 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae6eea5-1941-43a4-84ca-a58e99c17a46 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:47.088706 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae6eea5-1941-43a4-84ca-a58e99c17a46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:47.089113 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae6eea5-1941-43a4-84ca-a58e99c17a46 admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:47.089950 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae6eea5-1941-43a4-84ca-a58e99c17a46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:47.092192 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae6eea5-1941-43a4-84ca-a58e99c17a46 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:47.112298 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1171/2347] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 17 19:31:46 2022] POST /identity/v3/projects => generated 377 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:31:47.117499 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467ca133-8575-4a17-96dd-e78ba2b346b4 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:47.202610 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1177/2348] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 17 19:31:47 2022] POST /identity/v3/projects => generated 369 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 17 19:31:47.202610 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae6eea5-1941-43a4-84ca-a58e99c17a46 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:47.213601 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467ca133-8575-4a17-96dd-e78ba2b346b4 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:47.311861 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654171f1-e508-4135-b965-38b7dbbbf990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:47.312559 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654171f1-e508-4135-b965-38b7dbbbf990 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:47.312983 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654171f1-e508-4135-b965-38b7dbbbf990 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:47.313265 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654171f1-e508-4135-b965-38b7dbbbf990 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:47.314011 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654171f1-e508-4135-b965-38b7dbbbf990 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:47.321241 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654171f1-e508-4135-b965-38b7dbbbf990 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:47.364580 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae6eea5-1941-43a4-84ca-a58e99c17a46 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:47.374396 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbe5afb-a7da-489c-bf41-cf7999626a80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:47.378815 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbe5afb-a7da-489c-bf41-cf7999626a80 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:47.379101 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbe5afb-a7da-489c-bf41-cf7999626a80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:47.383837 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbe5afb-a7da-489c-bf41-cf7999626a80 admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:47.385829 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbe5afb-a7da-489c-bf41-cf7999626a80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:47.388715 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbe5afb-a7da-489c-bf41-cf7999626a80 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:47.422008 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1172/2349] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 17 19:31:47 2022] POST /identity/v3/users => generated 289 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:31:47.426360 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654171f1-e508-4135-b965-38b7dbbbf990 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:47.441584 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654171f1-e508-4135-b965-38b7dbbbf990 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:47.446614 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3c4878-7fa5-43bf-b0d1-b000adc9141a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:47.448145 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3c4878-7fa5-43bf-b0d1-b000adc9141a admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:47.448426 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3c4878-7fa5-43bf-b0d1-b000adc9141a admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:47.449216 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3c4878-7fa5-43bf-b0d1-b000adc9141a admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:47.449937 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3c4878-7fa5-43bf-b0d1-b000adc9141a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:47.451853 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3c4878-7fa5-43bf-b0d1-b000adc9141a admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:47.463428 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1173/2350] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:31:47 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:31:47.468919 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3c4878-7fa5-43bf-b0d1-b000adc9141a admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:47.475355 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1178/2351] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 17 19:31:47 2022] POST /identity/v3/users => generated 285 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 17 19:31:47.479064 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbe5afb-a7da-489c-bf41-cf7999626a80 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:47.483038 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3c4878-7fa5-43bf-b0d1-b000adc9141a admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:47.490862 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7e140f-0117-4edb-9a24-3de91ce09bde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:47.491580 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7e140f-0117-4edb-9a24-3de91ce09bde admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:47.492153 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7e140f-0117-4edb-9a24-3de91ce09bde admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:47.492433 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7e140f-0117-4edb-9a24-3de91ce09bde admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:47.493408 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7e140f-0117-4edb-9a24-3de91ce09bde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:47.495098 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7e140f-0117-4edb-9a24-3de91ce09bde admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:47.495223 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbe5afb-a7da-489c-bf41-cf7999626a80 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:47.510111 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1174/2352] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:31:47 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:31:47.512186 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7e140f-0117-4edb-9a24-3de91ce09bde admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:47.514300 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de705ad-b0d7-47f4-91e2-f639d4da7f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:47.514300 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de705ad-b0d7-47f4-91e2-f639d4da7f20 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:47.514300 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de705ad-b0d7-47f4-91e2-f639d4da7f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:47.514300 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de705ad-b0d7-47f4-91e2-f639d4da7f20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:47.516338 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de705ad-b0d7-47f4-91e2-f639d4da7f20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:47.519981 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de705ad-b0d7-47f4-91e2-f639d4da7f20 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:47.527420 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7e140f-0117-4edb-9a24-3de91ce09bde admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:47.535100 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57df81b6-11b9-49fa-aee6-8b802d16609d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:47.536860 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57df81b6-11b9-49fa-aee6-8b802d16609d admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:47.537341 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57df81b6-11b9-49fa-aee6-8b802d16609d admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:47.537686 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57df81b6-11b9-49fa-aee6-8b802d16609d admin admin] PATH_INFO: `/v3/projects/aae36ceec2314da8a829113fc36f2f40/users/2e4f7fc1ab304418b892be060814bd8f/roles/114e6a16e63447d1924707ace5aace25` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:47.546315 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1179/2353] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:31:47 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:31:47.546905 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de705ad-b0d7-47f4-91e2-f639d4da7f20 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:47.553275 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-1de705ad-b0d7-47f4-91e2-f639d4da7f20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:47.583309 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57df81b6-11b9-49fa-aee6-8b802d16609d admin admin] RBAC: Authorizing `identity:create_grant(project_id=aae36ceec2314da8a829113fc36f2f40, user_id=2e4f7fc1ab304418b892be060814bd8f, role_id=114e6a16e63447d1924707ace5aace25)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:47.590221 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57df81b6-11b9-49fa-aee6-8b802d16609d admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:47.605515 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de705ad-b0d7-47f4-91e2-f639d4da7f20 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:47.612239 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8ad6d9-dce6-4ab7-9d6d-14ec1caa824f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:47.613292 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8ad6d9-dce6-4ab7-9d6d-14ec1caa824f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:47.613773 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8ad6d9-dce6-4ab7-9d6d-14ec1caa824f admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:47.614048 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8ad6d9-dce6-4ab7-9d6d-14ec1caa824f admin admin] PATH_INFO: `/v3/projects/913af32bc6644584b35aa5c1ae6ffabb` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:47.615575 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8ad6d9-dce6-4ab7-9d6d-14ec1caa824f admin admin] RBAC: Authorizing `identity:delete_project(project_id=913af32bc6644584b35aa5c1ae6ffabb)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:47.618217 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8ad6d9-dce6-4ab7-9d6d-14ec1caa824f admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:47.681736 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1175/2354] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:31:47 2022] PUT /identity/v3/projects/aae36ceec2314da8a829113fc36f2f40/users/2e4f7fc1ab304418b892be060814bd8f/roles/114e6a16e63447d1924707ace5aace25 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:31:47.683055 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57df81b6-11b9-49fa-aee6-8b802d16609d admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:47.789307 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57df81b6-11b9-49fa-aee6-8b802d16609d admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:47.827229 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-2c8ad6d9-dce6-4ab7-9d6d-14ec1caa824f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '913af32bc6644584b35aa5c1ae6ffabb', 'request_id': 'req-2c8ad6d9-dce6-4ab7-9d6d-14ec1caa824f'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:47.837729 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-2c8ad6d9-dce6-4ab7-9d6d-14ec1caa824f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '913af32bc6644584b35aa5c1ae6ffabb', 'request_id': 'req-2c8ad6d9-dce6-4ab7-9d6d-14ec1caa824f'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:47.851533 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-2c8ad6d9-dce6-4ab7-9d6d-14ec1caa824f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '913af32bc6644584b35aa5c1ae6ffabb', 'request_id': 'req-2c8ad6d9-dce6-4ab7-9d6d-14ec1caa824f'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:47.899395 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7f884c-4958-4664-b51c-47d6d564ca3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:47.900160 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7f884c-4958-4664-b51c-47d6d564ca3f admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:47.900446 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7f884c-4958-4664-b51c-47d6d564ca3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:47.900713 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7f884c-4958-4664-b51c-47d6d564ca3f admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:47.905594 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7f884c-4958-4664-b51c-47d6d564ca3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:47.906184 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7f884c-4958-4664-b51c-47d6d564ca3f admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:47.919142 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1176/2355] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:31:47 2022] GET /identity/v3/roles => generated 1420 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:31:47.921843 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4edb32-3e0f-4c49-823b-d489fd43d863 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:47.922291 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4edb32-3e0f-4c49-823b-d489fd43d863 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:47.926382 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4edb32-3e0f-4c49-823b-d489fd43d863 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:47.931934 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-2c8ad6d9-dce6-4ab7-9d6d-14ec1caa824f admin admin] The token cache is being invalidate because project 913af32bc6644584b35aa5c1ae6ffabb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 17 19:31:47.932615 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-2c8ad6d9-dce6-4ab7-9d6d-14ec1caa824f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:47.935410 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-2c8ad6d9-dce6-4ab7-9d6d-14ec1caa824f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:47.936216 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1180/2356] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 17 19:31:47 2022] DELETE /identity/v3/projects/913af32bc6644584b35aa5c1ae6ffabb => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:31:47.941882 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8ad6d9-dce6-4ab7-9d6d-14ec1caa824f admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:47.965436 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-2c8ad6d9-dce6-4ab7-9d6d-14ec1caa824f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:47.969602 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-7f4edb32-3e0f-4c49-823b-d489fd43d863 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:31:47.971060 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-7f4edb32-3e0f-4c49-823b-d489fd43d863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:47.981539 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1177/2357] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:31:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:31:47.986876 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9defb302-66d9-42ab-b086-cbef80b25bbc None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:47.987124 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9defb302-66d9-42ab-b086-cbef80b25bbc None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:47.987511 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9defb302-66d9-42ab-b086-cbef80b25bbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:47.998429 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8ad6d9-dce6-4ab7-9d6d-14ec1caa824f admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:48.007459 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64faa019-a412-4fea-a049-020252067f49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:48.008653 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64faa019-a412-4fea-a049-020252067f49 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:48.008934 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64faa019-a412-4fea-a049-020252067f49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:48.009201 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64faa019-a412-4fea-a049-020252067f49 admin admin] PATH_INFO: `/v3/projects/43709e6b4496459e99b07296a3bea2e9/users/50392e3422f0473db858ed04379f193b/roles/114e6a16e63447d1924707ace5aace25` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:48.015983 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-9defb302-66d9-42ab-b086-cbef80b25bbc None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:31:48.017274 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-9defb302-66d9-42ab-b086-cbef80b25bbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:48.027352 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64faa019-a412-4fea-a049-020252067f49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43709e6b4496459e99b07296a3bea2e9, user_id=50392e3422f0473db858ed04379f193b, role_id=114e6a16e63447d1924707ace5aace25)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:48.034222 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1178/2358] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:31:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:31:48.034222 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9defb302-66d9-42ab-b086-cbef80b25bbc None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:48.034222 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-9defb302-66d9-42ab-b086-cbef80b25bbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:48.035541 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64faa019-a412-4fea-a049-020252067f49 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:48.093330 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9defb302-66d9-42ab-b086-cbef80b25bbc None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:48.097305 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a588d83e-ab65-43ab-95f4-4e162ba2d3e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:48.098398 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a588d83e-ab65-43ab-95f4-4e162ba2d3e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:48.098687 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a588d83e-ab65-43ab-95f4-4e162ba2d3e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:48.098962 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a588d83e-ab65-43ab-95f4-4e162ba2d3e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:48.109259 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a588d83e-ab65-43ab-95f4-4e162ba2d3e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:48.110946 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a588d83e-ab65-43ab-95f4-4e162ba2d3e9 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:48.123602 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1179/2359] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:31:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:31:48.125024 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d722b067-6e25-4e69-a2ca-782c759138ac None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:48.125437 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d722b067-6e25-4e69-a2ca-782c759138ac None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:48.125813 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d722b067-6e25-4e69-a2ca-782c759138ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:48.137771 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1181/2360] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:31:47 2022] PUT /identity/v3/projects/43709e6b4496459e99b07296a3bea2e9/users/50392e3422f0473db858ed04379f193b/roles/114e6a16e63447d1924707ace5aace25 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:31:48.137771 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc56f97e-e14a-4de3-a6fc-b7a0c2d73f3c None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:48.137771 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc56f97e-e14a-4de3-a6fc-b7a0c2d73f3c None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:48.137771 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc56f97e-e14a-4de3-a6fc-b7a0c2d73f3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:48.156345 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-fc56f97e-e14a-4de3-a6fc-b7a0c2d73f3c None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:31:48.158643 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-fc56f97e-e14a-4de3-a6fc-b7a0c2d73f3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:48.178781 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-d722b067-6e25-4e69-a2ca-782c759138ac None None] MFA Rules not processed for user `2b90b57c542440448579b3aadda8b241`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:31:48.180361 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d722b067-6e25-4e69-a2ca-782c759138ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:48.232309 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1182/2361] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:31:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:31:48.233492 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144dc9d1-8224-4206-be5a-75708e9e7cc7 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:48.233772 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144dc9d1-8224-4206-be5a-75708e9e7cc7 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:48.237126 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144dc9d1-8224-4206-be5a-75708e9e7cc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:48.251214 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1180/2362] 10.222.0.58 () {58 vars in 1009 bytes} [Tue May 17 19:31:48 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:31:48.252248 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d722b067-6e25-4e69-a2ca-782c759138ac None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:48.270844 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d722b067-6e25-4e69-a2ca-782c759138ac None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:48.274613 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-144dc9d1-8224-4206-be5a-75708e9e7cc7 None None] MFA Rules not processed for user `50392e3422f0473db858ed04379f193b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:31:48.275932 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-144dc9d1-8224-4206-be5a-75708e9e7cc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:48.277784 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214801f0-940b-4bcd-b1ca-053cacac6f66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:48.278440 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214801f0-940b-4bcd-b1ca-053cacac6f66 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:48.278720 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214801f0-940b-4bcd-b1ca-053cacac6f66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:48.278989 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214801f0-940b-4bcd-b1ca-053cacac6f66 admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:48.279745 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214801f0-940b-4bcd-b1ca-053cacac6f66 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:31:48.280045 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214801f0-940b-4bcd-b1ca-053cacac6f66 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:48.281717 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214801f0-940b-4bcd-b1ca-053cacac6f66 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:48.293958 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1181/2363] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 17 19:31:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:31:48.298392 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214801f0-940b-4bcd-b1ca-053cacac6f66 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:48.315746 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214801f0-940b-4bcd-b1ca-053cacac6f66 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:48.316102 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257a3d7f-23a5-4ed2-9bdc-755a5dcf6270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:48.316757 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257a3d7f-23a5-4ed2-9bdc-755a5dcf6270 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:48.317032 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257a3d7f-23a5-4ed2-9bdc-755a5dcf6270 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:48.317300 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257a3d7f-23a5-4ed2-9bdc-755a5dcf6270 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:48.318007 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257a3d7f-23a5-4ed2-9bdc-755a5dcf6270 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:48.319953 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257a3d7f-23a5-4ed2-9bdc-755a5dcf6270 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:48.357599 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1183/2364] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:31:48 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:31:48.358446 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144dc9d1-8224-4206-be5a-75708e9e7cc7 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:48.359734 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-144dc9d1-8224-4206-be5a-75708e9e7cc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:48.446810 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1182/2365] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 17 19:31:48 2022] POST /identity/v3/projects => generated 371 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 17 19:31:48.446810 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257a3d7f-23a5-4ed2-9bdc-755a5dcf6270 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:48.451673 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-257a3d7f-23a5-4ed2-9bdc-755a5dcf6270 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:48.456937 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144dc9d1-8224-4206-be5a-75708e9e7cc7 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:48.533507 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257a3d7f-23a5-4ed2-9bdc-755a5dcf6270 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:48.538385 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9b4ca8-2a24-4311-95d7-0e722015c8ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:48.539297 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9b4ca8-2a24-4311-95d7-0e722015c8ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:48.539606 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9b4ca8-2a24-4311-95d7-0e722015c8ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:48.539705 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4351a827-16a0-4b76-b9f2-08e82521f2f8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0cb292002434d30bc81f370d120a2a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:48.539862 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9b4ca8-2a24-4311-95d7-0e722015c8ee admin admin] PATH_INFO: `/v3/users/7eacc38176944eedbd178e43b1e5d035` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:48.540706 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4351a827-16a0-4b76-b9f2-08e82521f2f8 service swift] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:48.541006 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4351a827-16a0-4b76-b9f2-08e82521f2f8 service swift] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:48.541287 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4351a827-16a0-4b76-b9f2-08e82521f2f8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:48.544563 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9b4ca8-2a24-4311-95d7-0e722015c8ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=7eacc38176944eedbd178e43b1e5d035)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:48.546500 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9b4ca8-2a24-4311-95d7-0e722015c8ee admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:48.601978 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4351a827-16a0-4b76-b9f2-08e82521f2f8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:48.603604 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4351a827-16a0-4b76-b9f2-08e82521f2f8 service swift] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:48.611711 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1184/2366] 10.222.0.58 () {60 vars in 1384 bytes} [Tue May 17 19:31:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 17 19:31:48.612474 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4351a827-16a0-4b76-b9f2-08e82521f2f8 service swift] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:48.633300 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4351a827-16a0-4b76-b9f2-08e82521f2f8 service swift] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:48.638919 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb54727-9359-472b-87ef-42dbcb8b3e62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:48.639736 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb54727-9359-472b-87ef-42dbcb8b3e62 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:48.640201 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb54727-9359-472b-87ef-42dbcb8b3e62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:48.640441 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb54727-9359-472b-87ef-42dbcb8b3e62 admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:48.641024 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb54727-9359-472b-87ef-42dbcb8b3e62 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:48.643006 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb54727-9359-472b-87ef-42dbcb8b3e62 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:48.757402 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1185/2367] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 17 19:31:48 2022] POST /identity/v3/users => generated 286 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:31:48.762423 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb54727-9359-472b-87ef-42dbcb8b3e62 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:48.792858 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb54727-9359-472b-87ef-42dbcb8b3e62 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:48.801860 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef54926-3606-4f0b-8676-a9da4d1a0e35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:48.801860 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef54926-3606-4f0b-8676-a9da4d1a0e35 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:48.801860 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef54926-3606-4f0b-8676-a9da4d1a0e35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:48.801860 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef54926-3606-4f0b-8676-a9da4d1a0e35 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:48.802339 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef54926-3606-4f0b-8676-a9da4d1a0e35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:48.803901 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef54926-3606-4f0b-8676-a9da4d1a0e35 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:48.823120 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1186/2368] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:31:48 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:31:48.825901 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ae9b4ca8-2a24-4311-95d7-0e722015c8ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7eacc38176944eedbd178e43b1e5d035'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:48.829262 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ae9b4ca8-2a24-4311-95d7-0e722015c8ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7eacc38176944eedbd178e43b1e5d035'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:48.831983 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef54926-3606-4f0b-8676-a9da4d1a0e35 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:48.836134 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-3ef54926-3606-4f0b-8676-a9da4d1a0e35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:48.849150 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ae9b4ca8-2a24-4311-95d7-0e722015c8ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7eacc38176944eedbd178e43b1e5d035'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:48.853185 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef54926-3606-4f0b-8676-a9da4d1a0e35 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:48.861675 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6c31c5-581a-4a71-a20d-45abbb024d53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:48.862500 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6c31c5-581a-4a71-a20d-45abbb024d53 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:48.865808 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6c31c5-581a-4a71-a20d-45abbb024d53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:48.865808 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6c31c5-581a-4a71-a20d-45abbb024d53 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:48.866378 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6c31c5-581a-4a71-a20d-45abbb024d53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:48.868316 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6c31c5-581a-4a71-a20d-45abbb024d53 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:48.882254 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1187/2369] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:31:48 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:31:48.887017 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6c31c5-581a-4a71-a20d-45abbb024d53 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:48.898633 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6c31c5-581a-4a71-a20d-45abbb024d53 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:48.908761 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f5a7ec-51c2-4e9f-8649-52822a02a02a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:48.909671 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f5a7ec-51c2-4e9f-8649-52822a02a02a admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:48.909950 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f5a7ec-51c2-4e9f-8649-52822a02a02a admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:48.910225 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f5a7ec-51c2-4e9f-8649-52822a02a02a admin admin] PATH_INFO: `/v3/projects/526f0c023217424d8ddde06706fff79c/users/34e5f89510c04ac78c6f3a94dddc23de/roles/114e6a16e63447d1924707ace5aace25` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:48.927952 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f5a7ec-51c2-4e9f-8649-52822a02a02a admin admin] RBAC: Authorizing `identity:create_grant(project_id=526f0c023217424d8ddde06706fff79c, user_id=34e5f89510c04ac78c6f3a94dddc23de, role_id=114e6a16e63447d1924707ace5aace25)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:48.930643 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f5a7ec-51c2-4e9f-8649-52822a02a02a admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:49.057478 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ae9b4ca8-2a24-4311-95d7-0e722015c8ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7eacc38176944eedbd178e43b1e5d035'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:49.065726 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ae9b4ca8-2a24-4311-95d7-0e722015c8ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7eacc38176944eedbd178e43b1e5d035'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:49.072318 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1183/2370] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 17 19:31:48 2022] DELETE /identity/v3/users/7eacc38176944eedbd178e43b1e5d035 => generated 0 bytes in 628 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:31:49.077255 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e47a9a-808b-4984-aade-2d5e6d4c0b4f None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:49.077698 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e47a9a-808b-4984-aade-2d5e6d4c0b4f None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:49.078094 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e47a9a-808b-4984-aade-2d5e6d4c0b4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:49.082955 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1188/2371] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:31:48 2022] PUT /identity/v3/projects/526f0c023217424d8ddde06706fff79c/users/34e5f89510c04ac78c6f3a94dddc23de/roles/114e6a16e63447d1924707ace5aace25 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:31:49.091830 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e35502-165e-4432-a466-d7941937d023 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:49.092142 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e35502-165e-4432-a466-d7941937d023 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:49.094912 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e35502-165e-4432-a466-d7941937d023 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:49.098772 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-f7e47a9a-808b-4984-aade-2d5e6d4c0b4f None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:31:49.101074 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-f7e47a9a-808b-4984-aade-2d5e6d4c0b4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:49.122991 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-38e35502-165e-4432-a466-d7941937d023 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:31:49.127322 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-38e35502-165e-4432-a466-d7941937d023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:49.248276 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1189/2372] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:31:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:31:49.248495 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1184/2373] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:31:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:31:49.259173 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e35502-165e-4432-a466-d7941937d023 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:49.262443 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-38e35502-165e-4432-a466-d7941937d023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:49.271921 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e47a9a-808b-4984-aade-2d5e6d4c0b4f None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:49.273352 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-f7e47a9a-808b-4984-aade-2d5e6d4c0b4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:49.375174 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e35502-165e-4432-a466-d7941937d023 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:49.379829 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d507d108-2a55-4ba8-a7cc-c11f785d5423 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:49.380625 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d507d108-2a55-4ba8-a7cc-c11f785d5423 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:49.380912 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d507d108-2a55-4ba8-a7cc-c11f785d5423 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:49.381190 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d507d108-2a55-4ba8-a7cc-c11f785d5423 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:49.404365 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e47a9a-808b-4984-aade-2d5e6d4c0b4f None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:49.408437 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d507d108-2a55-4ba8-a7cc-c11f785d5423 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:49.410434 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d507d108-2a55-4ba8-a7cc-c11f785d5423 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:49.418624 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822f3f84-98d6-4153-87ef-0a9eacefcc78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:49.419312 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822f3f84-98d6-4153-87ef-0a9eacefcc78 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:49.419719 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822f3f84-98d6-4153-87ef-0a9eacefcc78 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:49.420003 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822f3f84-98d6-4153-87ef-0a9eacefcc78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:49.440774 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822f3f84-98d6-4153-87ef-0a9eacefcc78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:49.442622 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822f3f84-98d6-4153-87ef-0a9eacefcc78 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:49.456052 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1190/2374] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:31:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:31:49.458117 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1185/2375] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:31:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 17 19:31:50.267865 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822f3f84-98d6-4153-87ef-0a9eacefcc78 service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:50.269412 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-822f3f84-98d6-4153-87ef-0a9eacefcc78 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:50.298296 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822f3f84-98d6-4153-87ef-0a9eacefcc78 service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:50.303100 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5203924e-15d2-491d-9dcf-81647d7495de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:50.304569 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5203924e-15d2-491d-9dcf-81647d7495de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:50.304569 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5203924e-15d2-491d-9dcf-81647d7495de admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:50.304782 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5203924e-15d2-491d-9dcf-81647d7495de admin admin] PATH_INFO: `/v3/projects/cd0a7a46d0d34dd480e771b2d711242a` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:50.305981 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5203924e-15d2-491d-9dcf-81647d7495de admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd0a7a46d0d34dd480e771b2d711242a)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:50.308104 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5203924e-15d2-491d-9dcf-81647d7495de admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:50.476178 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-5203924e-15d2-491d-9dcf-81647d7495de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd0a7a46d0d34dd480e771b2d711242a', 'request_id': 'req-5203924e-15d2-491d-9dcf-81647d7495de'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:50.484040 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-5203924e-15d2-491d-9dcf-81647d7495de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd0a7a46d0d34dd480e771b2d711242a', 'request_id': 'req-5203924e-15d2-491d-9dcf-81647d7495de'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:50.548980 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-5203924e-15d2-491d-9dcf-81647d7495de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd0a7a46d0d34dd480e771b2d711242a', 'request_id': 'req-5203924e-15d2-491d-9dcf-81647d7495de'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:50.582416 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-5203924e-15d2-491d-9dcf-81647d7495de admin admin] The token cache is being invalidate because project cd0a7a46d0d34dd480e771b2d711242a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 17 19:31:50.583106 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-5203924e-15d2-491d-9dcf-81647d7495de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:50.587467 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-5203924e-15d2-491d-9dcf-81647d7495de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:31:50.591720 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1186/2376] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 17 19:31:50 2022] DELETE /identity/v3/projects/cd0a7a46d0d34dd480e771b2d711242a => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:31:50.596670 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa2d86c-7696-44fd-8359-528dc7aec78e None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:50.597257 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa2d86c-7696-44fd-8359-528dc7aec78e None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:50.597257 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa2d86c-7696-44fd-8359-528dc7aec78e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:50.638697 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-baa2d86c-7696-44fd-8359-528dc7aec78e None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:31:50.639971 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-baa2d86c-7696-44fd-8359-528dc7aec78e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:50.768649 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1191/2377] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:31:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:31:50.776343 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2278981-402b-4895-bea3-e0d3c9c1e35c None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:50.778890 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2278981-402b-4895-bea3-e0d3c9c1e35c None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:50.779728 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2278981-402b-4895-bea3-e0d3c9c1e35c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:50.807718 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-f2278981-402b-4895-bea3-e0d3c9c1e35c None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:31:50.810424 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-f2278981-402b-4895-bea3-e0d3c9c1e35c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:50.824244 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1187/2378] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:31:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:31:50.828712 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa2d86c-7696-44fd-8359-528dc7aec78e None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:50.845007 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa2d86c-7696-44fd-8359-528dc7aec78e None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:50.859369 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645335a4-d2a1-474c-8246-d8348a01bc0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:50.860228 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645335a4-d2a1-474c-8246-d8348a01bc0f admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:50.862483 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645335a4-d2a1-474c-8246-d8348a01bc0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:50.863013 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645335a4-d2a1-474c-8246-d8348a01bc0f admin admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:50.864117 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645335a4-d2a1-474c-8246-d8348a01bc0f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:31:50.864458 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645335a4-d2a1-474c-8246-d8348a01bc0f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:50.870910 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645335a4-d2a1-474c-8246-d8348a01bc0f admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:50.900582 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1192/2379] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 17 19:31:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:31:50.914457 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2278981-402b-4895-bea3-e0d3c9c1e35c None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:50.934453 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2278981-402b-4895-bea3-e0d3c9c1e35c None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:50.940105 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d619ebb0-3a0f-40ba-b6bc-5da691833e8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:50.941027 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d619ebb0-3a0f-40ba-b6bc-5da691833e8b admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:50.941314 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d619ebb0-3a0f-40ba-b6bc-5da691833e8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:50.941682 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d619ebb0-3a0f-40ba-b6bc-5da691833e8b admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:50.942488 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d619ebb0-3a0f-40ba-b6bc-5da691833e8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:50.945005 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d619ebb0-3a0f-40ba-b6bc-5da691833e8b admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:51.142712 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1188/2380] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 17 19:31:50 2022] POST /identity/v3/projects => generated 399 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 17 19:31:51.157896 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645335a4-d2a1-474c-8246-d8348a01bc0f admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:51.183644 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645335a4-d2a1-474c-8246-d8348a01bc0f admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:51.356645 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254e8b0e-064e-426e-b812-616246b0931a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:51.356645 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254e8b0e-064e-426e-b812-616246b0931a admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:51.356645 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254e8b0e-064e-426e-b812-616246b0931a admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:51.356645 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254e8b0e-064e-426e-b812-616246b0931a admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:51.357195 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254e8b0e-064e-426e-b812-616246b0931a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:51.360307 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254e8b0e-064e-426e-b812-616246b0931a admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:51.494662 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1193/2381] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 17 19:31:51 2022] POST /identity/v3/users => generated 300 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:31:51.497849 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d619ebb0-3a0f-40ba-b6bc-5da691833e8b admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:51.513988 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d619ebb0-3a0f-40ba-b6bc-5da691833e8b admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:51.523207 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21f3dd0-ca7c-4abf-905c-4133c2264fc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:51.524394 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21f3dd0-ca7c-4abf-905c-4133c2264fc4 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:51.524872 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21f3dd0-ca7c-4abf-905c-4133c2264fc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:51.525346 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21f3dd0-ca7c-4abf-905c-4133c2264fc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:51.526257 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21f3dd0-ca7c-4abf-905c-4133c2264fc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:51.528623 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21f3dd0-ca7c-4abf-905c-4133c2264fc4 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:51.561026 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1189/2382] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:31:51 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 17 19:31:51.562300 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254e8b0e-064e-426e-b812-616246b0931a admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:51.577252 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254e8b0e-064e-426e-b812-616246b0931a admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:51.582933 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d2dca6-76ae-4b94-af37-bdf92e9d1af3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:51.583919 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d2dca6-76ae-4b94-af37-bdf92e9d1af3 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:51.584361 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d2dca6-76ae-4b94-af37-bdf92e9d1af3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:51.584731 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d2dca6-76ae-4b94-af37-bdf92e9d1af3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:51.586949 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d2dca6-76ae-4b94-af37-bdf92e9d1af3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:51.589335 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d2dca6-76ae-4b94-af37-bdf92e9d1af3 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:51.617174 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1194/2383] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:31:51 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:31:51.621754 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21f3dd0-ca7c-4abf-905c-4133c2264fc4 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:51.641116 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21f3dd0-ca7c-4abf-905c-4133c2264fc4 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:51.646422 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cafc131-8dbb-4770-adb5-b20847af2eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:51.647598 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cafc131-8dbb-4770-adb5-b20847af2eac admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:51.647775 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cafc131-8dbb-4770-adb5-b20847af2eac admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:51.648687 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cafc131-8dbb-4770-adb5-b20847af2eac admin admin] PATH_INFO: `/v3/projects/14363683d295428eb6455235fc145caf/users/38d6e707b46a47599a5cdd4f65c4497c/roles/114e6a16e63447d1924707ace5aace25` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:51.670612 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cafc131-8dbb-4770-adb5-b20847af2eac admin admin] RBAC: Authorizing `identity:create_grant(project_id=14363683d295428eb6455235fc145caf, user_id=38d6e707b46a47599a5cdd4f65c4497c, role_id=114e6a16e63447d1924707ace5aace25)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:51.673593 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cafc131-8dbb-4770-adb5-b20847af2eac admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:51.764398 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1190/2384] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:31:51 2022] PUT /identity/v3/projects/14363683d295428eb6455235fc145caf/users/38d6e707b46a47599a5cdd4f65c4497c/roles/114e6a16e63447d1924707ace5aace25 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:31:51.775163 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7cfc17-28d5-4ee2-bd76-28778af848d5 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:51.776300 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7cfc17-28d5-4ee2-bd76-28778af848d5 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:51.779464 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7cfc17-28d5-4ee2-bd76-28778af848d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:51.848586 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-ed7cfc17-28d5-4ee2-bd76-28778af848d5 None None] MFA Rules not processed for user `38d6e707b46a47599a5cdd4f65c4497c`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:31:51.852196 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-ed7cfc17-28d5-4ee2-bd76-28778af848d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:52.071077 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1195/2385] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:31:51 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:31:52.084734 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cafc131-8dbb-4770-adb5-b20847af2eac admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:52.088664 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-2cafc131-8dbb-4770-adb5-b20847af2eac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:52.118338 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6b3be0-d828-40e9-9587-0d99377983a8 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:52.118739 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6b3be0-d828-40e9-9587-0d99377983a8 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:52.118921 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6b3be0-d828-40e9-9587-0d99377983a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:52.208814 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-3b6b3be0-d828-40e9-9587-0d99377983a8 None None] MFA Rules not processed for user `34e5f89510c04ac78c6f3a94dddc23de`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:31:52.210059 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-3b6b3be0-d828-40e9-9587-0d99377983a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:52.297919 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cafc131-8dbb-4770-adb5-b20847af2eac admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:52.310242 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc6c2d7-bfbb-49a4-8c08-4378c3517a77 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd05409b38ae346458abfe41ebda1c59e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:52.310924 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc6c2d7-bfbb-49a4-8c08-4378c3517a77 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:52.311205 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc6c2d7-bfbb-49a4-8c08-4378c3517a77 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:52.311477 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc6c2d7-bfbb-49a4-8c08-4378c3517a77 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:52.332644 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc6c2d7-bfbb-49a4-8c08-4378c3517a77 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:52.335975 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc6c2d7-bfbb-49a4-8c08-4378c3517a77 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:52.366565 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1191/2386] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:31:52 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:31:52.399194 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1196/2387] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:31:52 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:31:53.520479 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc6c2d7-bfbb-49a4-8c08-4378c3517a77 service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:53.529927 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-7bc6c2d7-bfbb-49a4-8c08-4378c3517a77 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:53.582188 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc6c2d7-bfbb-49a4-8c08-4378c3517a77 service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:53.585911 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e195acd-a717-4b21-8156-7de20e877b57 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57e888884d2b4629bfd12d1034ce5be4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:53.586561 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e195acd-a717-4b21-8156-7de20e877b57 service glance] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:53.586836 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e195acd-a717-4b21-8156-7de20e877b57 service glance] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:53.587137 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e195acd-a717-4b21-8156-7de20e877b57 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:53.589781 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-4e195acd-a717-4b21-8156-7de20e877b57 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:53.671010 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e195acd-a717-4b21-8156-7de20e877b57 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:53.672757 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e195acd-a717-4b21-8156-7de20e877b57 service glance] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:53.691212 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1192/2388] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:31:53 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 17 19:31:53.816988 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6b3be0-d828-40e9-9587-0d99377983a8 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:53.834938 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6b3be0-d828-40e9-9587-0d99377983a8 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:53.843623 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256473c0-5ce1-4edc-a264-43a3d6e9bee6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd05409b38ae346458abfe41ebda1c59e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:53.843623 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256473c0-5ce1-4edc-a264-43a3d6e9bee6 service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:53.843623 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256473c0-5ce1-4edc-a264-43a3d6e9bee6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:53.843623 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256473c0-5ce1-4edc-a264-43a3d6e9bee6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:53.908359 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256473c0-5ce1-4edc-a264-43a3d6e9bee6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:53.918347 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256473c0-5ce1-4edc-a264-43a3d6e9bee6 service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:53.971918 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1197/2389] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:31:53 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:31:56.482347 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba73ff2-3fa1-4750-bcaa-b9b9d76608e3 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:56.482887 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba73ff2-3fa1-4750-bcaa-b9b9d76608e3 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:56.482887 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba73ff2-3fa1-4750-bcaa-b9b9d76608e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:56.518560 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-2ba73ff2-3fa1-4750-bcaa-b9b9d76608e3 None None] MFA Rules not processed for user `34e5f89510c04ac78c6f3a94dddc23de`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:31:56.522719 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-2ba73ff2-3fa1-4750-bcaa-b9b9d76608e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:56.553414 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1193/2390] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:31:56 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:31:57.458322 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a03bcf-e824-4fbd-afad-6c22ee5f53e5 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:57.458796 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a03bcf-e824-4fbd-afad-6c22ee5f53e5 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:57.459111 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a03bcf-e824-4fbd-afad-6c22ee5f53e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:57.534578 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-06a03bcf-e824-4fbd-afad-6c22ee5f53e5 None None] MFA Rules not processed for user `2e4f7fc1ab304418b892be060814bd8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:31:57.536021 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-06a03bcf-e824-4fbd-afad-6c22ee5f53e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:57.664658 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba73ff2-3fa1-4750-bcaa-b9b9d76608e3 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:57.666300 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-2ba73ff2-3fa1-4750-bcaa-b9b9d76608e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:57.668042 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1198/2391] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:31:57 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 17 19:31:57.757085 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba73ff2-3fa1-4750-bcaa-b9b9d76608e3 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:57.763682 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d45746-6330-44d3-8688-e0c2b84a1dd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:57.764614 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d45746-6330-44d3-8688-e0c2b84a1dd1 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:57.765146 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d45746-6330-44d3-8688-e0c2b84a1dd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:57.765662 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d45746-6330-44d3-8688-e0c2b84a1dd1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:57.767060 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d45746-6330-44d3-8688-e0c2b84a1dd1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:57.770838 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d45746-6330-44d3-8688-e0c2b84a1dd1 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:57.894971 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1194/2392] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 17 19:31:57 2022] POST /identity/v3/projects => generated 379 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 17 19:31:57.898975 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a03bcf-e824-4fbd-afad-6c22ee5f53e5 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:57.918801 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a03bcf-e824-4fbd-afad-6c22ee5f53e5 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:57.993679 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b960d95c-16fb-42c2-a4aa-09a3397e508d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:57.994354 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b960d95c-16fb-42c2-a4aa-09a3397e508d admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:57.994636 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b960d95c-16fb-42c2-a4aa-09a3397e508d admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:57.994911 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b960d95c-16fb-42c2-a4aa-09a3397e508d admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:57.995713 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b960d95c-16fb-42c2-a4aa-09a3397e508d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:57.997673 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b960d95c-16fb-42c2-a4aa-09a3397e508d admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:58.131731 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1199/2393] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 17 19:31:57 2022] POST /identity/v3/users => generated 296 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:31:58.133054 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d45746-6330-44d3-8688-e0c2b84a1dd1 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:58.145052 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d45746-6330-44d3-8688-e0c2b84a1dd1 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:58.152259 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4633e8-54c6-4dc2-894b-2280be9b46f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:58.153198 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4633e8-54c6-4dc2-894b-2280be9b46f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:58.153645 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4633e8-54c6-4dc2-894b-2280be9b46f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:58.153920 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4633e8-54c6-4dc2-894b-2280be9b46f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:58.154666 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4633e8-54c6-4dc2-894b-2280be9b46f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:58.156386 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4633e8-54c6-4dc2-894b-2280be9b46f7 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:58.169027 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1195/2394] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:31:58 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:31:58.173510 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b960d95c-16fb-42c2-a4aa-09a3397e508d admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:58.185078 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b960d95c-16fb-42c2-a4aa-09a3397e508d admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:58.189885 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1addcc9-bfba-4dab-b00b-9a940d92eae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:58.190807 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1addcc9-bfba-4dab-b00b-9a940d92eae4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:58.191086 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1addcc9-bfba-4dab-b00b-9a940d92eae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:58.191362 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1addcc9-bfba-4dab-b00b-9a940d92eae4 admin admin] PATH_INFO: `/v3/projects/701b08622f74425e9cf481c9deebc111/users/61db6d0dfbef43969e2028a24462c360/roles/60af5088793e4379b88821a86552a344` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:58.209873 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1addcc9-bfba-4dab-b00b-9a940d92eae4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=701b08622f74425e9cf481c9deebc111, user_id=61db6d0dfbef43969e2028a24462c360, role_id=60af5088793e4379b88821a86552a344)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:58.216447 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1addcc9-bfba-4dab-b00b-9a940d92eae4 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:58.319687 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1200/2395] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:31:58 2022] PUT /identity/v3/projects/701b08622f74425e9cf481c9deebc111/users/61db6d0dfbef43969e2028a24462c360/roles/60af5088793e4379b88821a86552a344 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:31:58.326729 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4633e8-54c6-4dc2-894b-2280be9b46f7 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:58.357958 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4633e8-54c6-4dc2-894b-2280be9b46f7 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:58.459292 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890ead8a-5161-4258-807b-728fe1dd6770 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:58.459799 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890ead8a-5161-4258-807b-728fe1dd6770 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:58.460081 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890ead8a-5161-4258-807b-728fe1dd6770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:58.469035 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f23b97-96be-4264-b6e3-f3b5dbb8e8fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:58.474664 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f23b97-96be-4264-b6e3-f3b5dbb8e8fe admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:58.474664 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f23b97-96be-4264-b6e3-f3b5dbb8e8fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:58.474664 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f23b97-96be-4264-b6e3-f3b5dbb8e8fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:58.474664 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f23b97-96be-4264-b6e3-f3b5dbb8e8fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:58.474664 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f23b97-96be-4264-b6e3-f3b5dbb8e8fe admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:58.483658 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-890ead8a-5161-4258-807b-728fe1dd6770 None None] MFA Rules not processed for user `b171cf633e6e437fb509857b0a2962e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:31:58.485918 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-890ead8a-5161-4258-807b-728fe1dd6770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:31:58.515435 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1196/2396] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:31:58 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:31:58.516590 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f23b97-96be-4264-b6e3-f3b5dbb8e8fe admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:31:58.530402 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f23b97-96be-4264-b6e3-f3b5dbb8e8fe admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:31:58.535839 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3930008-7cbe-40dd-bb31-16046b145abb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:31:58.536975 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3930008-7cbe-40dd-bb31-16046b145abb admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:31:58.537666 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3930008-7cbe-40dd-bb31-16046b145abb admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:31:58.538095 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3930008-7cbe-40dd-bb31-16046b145abb admin admin] PATH_INFO: `/v3/projects/701b08622f74425e9cf481c9deebc111/users/61db6d0dfbef43969e2028a24462c360/roles/60af5088793e4379b88821a86552a344` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:31:58.541088 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3930008-7cbe-40dd-bb31-16046b145abb admin admin] RBAC: Authorizing `identity:create_grant(project_id=701b08622f74425e9cf481c9deebc111, user_id=61db6d0dfbef43969e2028a24462c360, role_id=60af5088793e4379b88821a86552a344)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:31:58.544703 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3930008-7cbe-40dd-bb31-16046b145abb admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:31:58.568401 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1197/2397] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:31:58 2022] PUT /identity/v3/projects/701b08622f74425e9cf481c9deebc111/users/61db6d0dfbef43969e2028a24462c360/roles/60af5088793e4379b88821a86552a344 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:31:58.586885 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1201/2398] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:31:58 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:32:05.329644 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3930008-7cbe-40dd-bb31-16046b145abb admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:32:05.392198 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3930008-7cbe-40dd-bb31-16046b145abb admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:32:05.396807 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ffd561-dbd2-4e9b-8357-7cc6c2161965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:32:05.397675 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ffd561-dbd2-4e9b-8357-7cc6c2161965 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:32:05.397950 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ffd561-dbd2-4e9b-8357-7cc6c2161965 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:32:05.398219 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ffd561-dbd2-4e9b-8357-7cc6c2161965 admin admin] PATH_INFO: `/v3/users/38d6e707b46a47599a5cdd4f65c4497c` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:32:05.399402 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ffd561-dbd2-4e9b-8357-7cc6c2161965 admin admin] RBAC: Authorizing `identity:delete_user(user_id=38d6e707b46a47599a5cdd4f65c4497c)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:32:05.401957 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ffd561-dbd2-4e9b-8357-7cc6c2161965 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:32:05.700520 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d9ffd561-dbd2-4e9b-8357-7cc6c2161965 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38d6e707b46a47599a5cdd4f65c4497c'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:32:05.701414 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d9ffd561-dbd2-4e9b-8357-7cc6c2161965 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38d6e707b46a47599a5cdd4f65c4497c'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:32:05.746635 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d9ffd561-dbd2-4e9b-8357-7cc6c2161965 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38d6e707b46a47599a5cdd4f65c4497c'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:32:05.845348 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d9ffd561-dbd2-4e9b-8357-7cc6c2161965 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38d6e707b46a47599a5cdd4f65c4497c'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:32:05.862092 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d9ffd561-dbd2-4e9b-8357-7cc6c2161965 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38d6e707b46a47599a5cdd4f65c4497c'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:32:05.879639 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1198/2399] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 17 19:32:05 2022] DELETE /identity/v3/users/38d6e707b46a47599a5cdd4f65c4497c => generated 0 bytes in 550 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:32:05.885544 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acae021-3917-4344-9b04-cdd16a32e095 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:32:05.886697 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acae021-3917-4344-9b04-cdd16a32e095 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:32:05.888189 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acae021-3917-4344-9b04-cdd16a32e095 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:32:05.930637 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-3acae021-3917-4344-9b04-cdd16a32e095 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:32:05.932233 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-3acae021-3917-4344-9b04-cdd16a32e095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:32:06.094417 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1202/2400] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:32:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:32:06.105622 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ffd561-dbd2-4e9b-8357-7cc6c2161965 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:32:06.107354 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d9ffd561-dbd2-4e9b-8357-7cc6c2161965 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:32:06.246483 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ffd561-dbd2-4e9b-8357-7cc6c2161965 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:32:06.261840 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e698bf-47a9-42a4-9747-6f33d8180bb4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:32:06.262511 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e698bf-47a9-42a4-9747-6f33d8180bb4 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:32:06.262788 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e698bf-47a9-42a4-9747-6f33d8180bb4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:32:06.263057 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e698bf-47a9-42a4-9747-6f33d8180bb4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:32:06.282340 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e698bf-47a9-42a4-9747-6f33d8180bb4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:32:06.284070 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e698bf-47a9-42a4-9747-6f33d8180bb4 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:32:06.312763 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1199/2401] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:32:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:32:06.884338 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acae021-3917-4344-9b04-cdd16a32e095 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:32:06.891973 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-3acae021-3917-4344-9b04-cdd16a32e095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:32:06.939510 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acae021-3917-4344-9b04-cdd16a32e095 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:32:06.976726 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b47a150-2774-4a78-89ae-d6c7510630cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:32:06.977701 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b47a150-2774-4a78-89ae-d6c7510630cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:32:06.978082 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b47a150-2774-4a78-89ae-d6c7510630cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:32:06.978503 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b47a150-2774-4a78-89ae-d6c7510630cc admin admin] PATH_INFO: `/v3/projects/14363683d295428eb6455235fc145caf` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:32:06.980072 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b47a150-2774-4a78-89ae-d6c7510630cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=14363683d295428eb6455235fc145caf)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:32:06.982337 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b47a150-2774-4a78-89ae-d6c7510630cc admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:32:07.125638 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56632a58-fee9-46ad-9ca1-e7dcd89f2f78 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:32:07.126174 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56632a58-fee9-46ad-9ca1-e7dcd89f2f78 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:32:07.127765 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56632a58-fee9-46ad-9ca1-e7dcd89f2f78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:32:07.171985 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-56632a58-fee9-46ad-9ca1-e7dcd89f2f78 None None] MFA Rules not processed for user `61db6d0dfbef43969e2028a24462c360`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:32:07.173181 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-56632a58-fee9-46ad-9ca1-e7dcd89f2f78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:32:07.184468 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-9b47a150-2774-4a78-89ae-d6c7510630cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14363683d295428eb6455235fc145caf', 'request_id': 'req-9b47a150-2774-4a78-89ae-d6c7510630cc'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:32:07.200622 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-9b47a150-2774-4a78-89ae-d6c7510630cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14363683d295428eb6455235fc145caf', 'request_id': 'req-9b47a150-2774-4a78-89ae-d6c7510630cc'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:32:07.208960 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-9b47a150-2774-4a78-89ae-d6c7510630cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14363683d295428eb6455235fc145caf', 'request_id': 'req-9b47a150-2774-4a78-89ae-d6c7510630cc'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:32:07.316165 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1200/2402] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:32:07 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:32:07.332108 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-9b47a150-2774-4a78-89ae-d6c7510630cc admin admin] The token cache is being invalidate because project 14363683d295428eb6455235fc145caf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 17 19:32:07.332108 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-9b47a150-2774-4a78-89ae-d6c7510630cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:32:07.332108 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-9b47a150-2774-4a78-89ae-d6c7510630cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:32:07.332108 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1203/2403] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 17 19:32:06 2022] DELETE /identity/v3/projects/14363683d295428eb6455235fc145caf => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:32:08.364580 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56632a58-fee9-46ad-9ca1-e7dcd89f2f78 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:32:08.366295 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-56632a58-fee9-46ad-9ca1-e7dcd89f2f78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:32:08.386425 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b47a150-2774-4a78-89ae-d6c7510630cc admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:32:08.388423 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-9b47a150-2774-4a78-89ae-d6c7510630cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:32:08.517970 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b47a150-2774-4a78-89ae-d6c7510630cc admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:32:08.528393 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e7b2c8-825a-4a30-bd0b-03c79638847c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:32:08.532431 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e7b2c8-825a-4a30-bd0b-03c79638847c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:32:08.534557 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e7b2c8-825a-4a30-bd0b-03c79638847c admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:32:08.535104 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e7b2c8-825a-4a30-bd0b-03c79638847c admin admin] PATH_INFO: `/v3/users/910494d191cc4e939c67dd546a914c08` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:32:08.539618 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e7b2c8-825a-4a30-bd0b-03c79638847c admin admin] RBAC: Authorizing `identity:delete_user(user_id=910494d191cc4e939c67dd546a914c08)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:32:08.543401 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e7b2c8-825a-4a30-bd0b-03c79638847c admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:32:08.556926 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56632a58-fee9-46ad-9ca1-e7dcd89f2f78 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:32:08.566299 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aae143a-9165-4573-b912-34fb36da6266 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c1f878b0dc4d63a4f5dcb12fac1dc1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:32:08.567013 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aae143a-9165-4573-b912-34fb36da6266 service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:32:08.567471 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aae143a-9165-4573-b912-34fb36da6266 service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:32:08.568011 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aae143a-9165-4573-b912-34fb36da6266 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:32:08.570047 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-2aae143a-9165-4573-b912-34fb36da6266 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:32:08.641606 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aae143a-9165-4573-b912-34fb36da6266 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:32:08.643433 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aae143a-9165-4573-b912-34fb36da6266 service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:32:08.654782 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1201/2404] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:32:08 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:32:08.798621 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-91e7b2c8-825a-4a30-bd0b-03c79638847c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '910494d191cc4e939c67dd546a914c08'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:32:08.806585 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-91e7b2c8-825a-4a30-bd0b-03c79638847c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '910494d191cc4e939c67dd546a914c08'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:32:08.824077 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-91e7b2c8-825a-4a30-bd0b-03c79638847c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '910494d191cc4e939c67dd546a914c08'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:32:08.826796 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-91e7b2c8-825a-4a30-bd0b-03c79638847c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '910494d191cc4e939c67dd546a914c08'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:32:08.827894 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-91e7b2c8-825a-4a30-bd0b-03c79638847c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '910494d191cc4e939c67dd546a914c08'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:32:08.925932 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1204/2405] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 17 19:32:08 2022] DELETE /identity/v3/users/910494d191cc4e939c67dd546a914c08 => generated 0 bytes in 541 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:32:08.930732 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3d2116-3967-43d7-b2c2-1f893f585c2c None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:32:08.931023 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3d2116-3967-43d7-b2c2-1f893f585c2c None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:32:08.931321 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3d2116-3967-43d7-b2c2-1f893f585c2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:32:08.974206 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-5c3d2116-3967-43d7-b2c2-1f893f585c2c None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:32:08.975573 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-5c3d2116-3967-43d7-b2c2-1f893f585c2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:32:09.111040 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1202/2406] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:32:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:32:09.131560 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e7b2c8-825a-4a30-bd0b-03c79638847c admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:32:09.133837 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-91e7b2c8-825a-4a30-bd0b-03c79638847c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:32:09.206904 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e7b2c8-825a-4a30-bd0b-03c79638847c admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:32:09.211230 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae5edfd-3da2-4063-a84c-ca461bc0d9f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:32:09.211913 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae5edfd-3da2-4063-a84c-ca461bc0d9f0 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:32:09.212195 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae5edfd-3da2-4063-a84c-ca461bc0d9f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:32:09.212497 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae5edfd-3da2-4063-a84c-ca461bc0d9f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:32:09.222906 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae5edfd-3da2-4063-a84c-ca461bc0d9f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:32:09.224958 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae5edfd-3da2-4063-a84c-ca461bc0d9f0 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:32:09.236067 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1205/2407] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:32:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:32:09.784613 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3d2116-3967-43d7-b2c2-1f893f585c2c None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:32:09.786868 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-5c3d2116-3967-43d7-b2c2-1f893f585c2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:32:09.800259 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3d2116-3967-43d7-b2c2-1f893f585c2c None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:32:09.807111 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab96bb2-29ed-4a8e-82ae-c338b204abff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:32:09.807210 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab96bb2-29ed-4a8e-82ae-c338b204abff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:32:09.808124 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab96bb2-29ed-4a8e-82ae-c338b204abff admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:32:09.808383 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab96bb2-29ed-4a8e-82ae-c338b204abff admin admin] PATH_INFO: `/v3/projects/83c30e8ab6e2481ca4dd41bc68d93846` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:32:09.810935 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab96bb2-29ed-4a8e-82ae-c338b204abff admin admin] RBAC: Authorizing `identity:delete_project(project_id=83c30e8ab6e2481ca4dd41bc68d93846)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:32:09.820577 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab96bb2-29ed-4a8e-82ae-c338b204abff admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:32:09.980402 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-7ab96bb2-29ed-4a8e-82ae-c338b204abff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83c30e8ab6e2481ca4dd41bc68d93846', 'request_id': 'req-7ab96bb2-29ed-4a8e-82ae-c338b204abff'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:32:09.986857 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-7ab96bb2-29ed-4a8e-82ae-c338b204abff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83c30e8ab6e2481ca4dd41bc68d93846', 'request_id': 'req-7ab96bb2-29ed-4a8e-82ae-c338b204abff'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:32:10.085093 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-7ab96bb2-29ed-4a8e-82ae-c338b204abff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83c30e8ab6e2481ca4dd41bc68d93846', 'request_id': 'req-7ab96bb2-29ed-4a8e-82ae-c338b204abff'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:32:10.111749 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-7ab96bb2-29ed-4a8e-82ae-c338b204abff admin admin] The token cache is being invalidate because project 83c30e8ab6e2481ca4dd41bc68d93846 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 17 19:32:10.111749 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-7ab96bb2-29ed-4a8e-82ae-c338b204abff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:32:10.111749 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-7ab96bb2-29ed-4a8e-82ae-c338b204abff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:32:10.111749 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1203/2408] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 17 19:32:09 2022] DELETE /identity/v3/projects/83c30e8ab6e2481ca4dd41bc68d93846 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 17 19:32:14.599901 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d8f36b-c657-4f9b-a148-8ac8f3f7f734 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:32:14.600437 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d8f36b-c657-4f9b-a148-8ac8f3f7f734 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:32:14.600610 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d8f36b-c657-4f9b-a148-8ac8f3f7f734 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:32:14.602056 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1206/2409] 10.222.0.58 () {54 vars in 921 bytes} [Tue May 17 19:32:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:32:14.609324 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb592fb-c730-406b-9e41-9b4b7eeb2976 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:32:14.609324 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb592fb-c730-406b-9e41-9b4b7eeb2976 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:32:14.609929 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb592fb-c730-406b-9e41-9b4b7eeb2976 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:32:14.684084 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-1cb592fb-c730-406b-9e41-9b4b7eeb2976 None None] MFA Rules not processed for user `65c1f878b0dc4d63a4f5dcb12fac1dc1`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:32:14.685509 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-1cb592fb-c730-406b-9e41-9b4b7eeb2976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:32:14.782054 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1204/2410] 10.222.0.58 () {58 vars in 1018 bytes} [Tue May 17 19:32:14 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:32:22.558725 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d8f36b-c657-4f9b-a148-8ac8f3f7f734 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:32:22.560537 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-c8d8f36b-c657-4f9b-a148-8ac8f3f7f734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:32:22.638244 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d8f36b-c657-4f9b-a148-8ac8f3f7f734 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:32:22.641737 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4c94d6-c6d9-44c2-8c5c-d8d52fbc9369 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57e888884d2b4629bfd12d1034ce5be4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:32:22.642378 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4c94d6-c6d9-44c2-8c5c-d8d52fbc9369 service glance] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:32:22.642652 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4c94d6-c6d9-44c2-8c5c-d8d52fbc9369 service glance] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:32:22.642919 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4c94d6-c6d9-44c2-8c5c-d8d52fbc9369 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:32:22.654057 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4c94d6-c6d9-44c2-8c5c-d8d52fbc9369 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:32:22.655534 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4c94d6-c6d9-44c2-8c5c-d8d52fbc9369 service glance] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:32:22.665877 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1207/2411] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:32:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:32:22.878470 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2898cce-626a-473e-bf5d-c6eb6b11aaa9 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:32:22.878997 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2898cce-626a-473e-bf5d-c6eb6b11aaa9 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:32:22.878997 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2898cce-626a-473e-bf5d-c6eb6b11aaa9 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:32:22.880395 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1205/2412] 10.222.0.58 () {54 vars in 921 bytes} [Tue May 17 19:32:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:32:22.885195 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eee8fce-e773-48dd-bf29-cd7cad38d0db None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:32:22.885484 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eee8fce-e773-48dd-bf29-cd7cad38d0db None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:32:22.885763 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eee8fce-e773-48dd-bf29-cd7cad38d0db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:32:22.904236 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-4eee8fce-e773-48dd-bf29-cd7cad38d0db None None] MFA Rules not processed for user `1ac193d15f48463c98ff1d97770ed794`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:32:22.905363 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-4eee8fce-e773-48dd-bf29-cd7cad38d0db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:32:22.957961 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1208/2413] 10.222.0.58 () {58 vars in 1018 bytes} [Tue May 17 19:32:22 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 17 19:32:22.970514 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2898cce-626a-473e-bf5d-c6eb6b11aaa9 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:32:22.972314 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-e2898cce-626a-473e-bf5d-c6eb6b11aaa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:32:22.989506 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2898cce-626a-473e-bf5d-c6eb6b11aaa9 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:32:22.995318 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e211827-2344-4297-b86a-23b71929c2cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:32:22.996010 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e211827-2344-4297-b86a-23b71929c2cc service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:32:22.996307 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e211827-2344-4297-b86a-23b71929c2cc service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:32:22.996566 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e211827-2344-4297-b86a-23b71929c2cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:32:23.006676 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e211827-2344-4297-b86a-23b71929c2cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:32:23.008317 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e211827-2344-4297-b86a-23b71929c2cc service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:32:23.019465 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1206/2414] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:32:22 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:32:23.101169 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660c23a3-45e0-4edf-a85a-9d645f3176df None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:32:23.101457 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660c23a3-45e0-4edf-a85a-9d645f3176df None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:32:23.101731 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660c23a3-45e0-4edf-a85a-9d645f3176df None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:32:23.103098 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1209/2415] 10.222.0.58 () {54 vars in 921 bytes} [Tue May 17 19:32:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 17 19:32:23.109080 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2338b21-c202-41b3-96ba-4fe3e4d58221 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:32:23.109460 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2338b21-c202-41b3-96ba-4fe3e4d58221 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:32:23.109801 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2338b21-c202-41b3-96ba-4fe3e4d58221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:32:23.130109 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-b2338b21-c202-41b3-96ba-4fe3e4d58221 None None] MFA Rules not processed for user `65c1f878b0dc4d63a4f5dcb12fac1dc1`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:32:23.131292 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b2338b21-c202-41b3-96ba-4fe3e4d58221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:32:23.141405 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1207/2416] 10.222.0.58 () {58 vars in 1018 bytes} [Tue May 17 19:32:23 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 17 19:32:23.152953 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660c23a3-45e0-4edf-a85a-9d645f3176df None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:32:23.155024 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-660c23a3-45e0-4edf-a85a-9d645f3176df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:32:23.169754 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660c23a3-45e0-4edf-a85a-9d645f3176df None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:32:23.173980 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f774c7f4-1a6a-4a70-9eee-7a9e5e06d93e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:32:23.174282 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f774c7f4-1a6a-4a70-9eee-7a9e5e06d93e service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:32:23.174507 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f774c7f4-1a6a-4a70-9eee-7a9e5e06d93e service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:32:23.174850 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f774c7f4-1a6a-4a70-9eee-7a9e5e06d93e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:32:23.186212 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f774c7f4-1a6a-4a70-9eee-7a9e5e06d93e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:32:23.187698 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f774c7f4-1a6a-4a70-9eee-7a9e5e06d93e service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:32:23.199346 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1210/2417] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:32:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:32:36.731077 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badede81-4679-4589-81aa-fafd9ae669a9 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:32:36.733618 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badede81-4679-4589-81aa-fafd9ae669a9 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:32:36.733618 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badede81-4679-4589-81aa-fafd9ae669a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:32:36.781388 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-badede81-4679-4589-81aa-fafd9ae669a9 None None] MFA Rules not processed for user `2b90b57c542440448579b3aadda8b241`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:32:36.786049 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-badede81-4679-4589-81aa-fafd9ae669a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:32:36.845254 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1208/2418] 10.222.0.58 () {58 vars in 1009 bytes} [Tue May 17 19:32:36 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:32:36.859417 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f774c7f4-1a6a-4a70-9eee-7a9e5e06d93e service neutron] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:32:36.861238 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-f774c7f4-1a6a-4a70-9eee-7a9e5e06d93e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:32:36.950401 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f774c7f4-1a6a-4a70-9eee-7a9e5e06d93e service neutron] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:32:36.955105 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419894ac-f962-481a-b20b-8fbc0ad288c9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0cb292002434d30bc81f370d120a2a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:32:36.955778 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419894ac-f962-481a-b20b-8fbc0ad288c9 service swift] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:32:36.956054 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419894ac-f962-481a-b20b-8fbc0ad288c9 service swift] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:32:36.956321 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419894ac-f962-481a-b20b-8fbc0ad288c9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:32:36.964552 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419894ac-f962-481a-b20b-8fbc0ad288c9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:32:36.968743 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419894ac-f962-481a-b20b-8fbc0ad288c9 service swift] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:32:36.973891 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1211/2419] 10.222.0.58 () {60 vars in 1384 bytes} [Tue May 17 19:32:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 17 19:32:37.251174 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c912bc96-bd0f-45e6-bde1-22626be918c2 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:32:37.251465 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c912bc96-bd0f-45e6-bde1-22626be918c2 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:32:37.251789 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c912bc96-bd0f-45e6-bde1-22626be918c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:32:37.272262 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-c912bc96-bd0f-45e6-bde1-22626be918c2 None None] MFA Rules not processed for user `34e5f89510c04ac78c6f3a94dddc23de`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:32:37.273464 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c912bc96-bd0f-45e6-bde1-22626be918c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:32:37.336176 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1209/2420] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:32:37 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:32:48.795987 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aabd12-df8a-403c-915c-da3faa2a3f95 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:32:48.796606 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aabd12-df8a-403c-915c-da3faa2a3f95 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:32:48.796606 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aabd12-df8a-403c-915c-da3faa2a3f95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:32:48.816040 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-a2aabd12-df8a-403c-915c-da3faa2a3f95 None None] MFA Rules not processed for user `2e4f7fc1ab304418b892be060814bd8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:32:48.817585 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-a2aabd12-df8a-403c-915c-da3faa2a3f95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:32:48.879470 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1212/2421] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:32:48 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:32:50.840758 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c912bc96-bd0f-45e6-bde1-22626be918c2 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:32:50.845197 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c912bc96-bd0f-45e6-bde1-22626be918c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:32:50.940563 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c912bc96-bd0f-45e6-bde1-22626be918c2 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:32:50.945711 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00a34a4-759a-4250-8b16-b19762b21d32 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06538aab5f194af7b7c677d6b94aef2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:32:50.946362 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00a34a4-759a-4250-8b16-b19762b21d32 service placement] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:32:50.946640 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00a34a4-759a-4250-8b16-b19762b21d32 service placement] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:32:50.946917 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00a34a4-759a-4250-8b16-b19762b21d32 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:32:50.949264 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a00a34a4-759a-4250-8b16-b19762b21d32 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:32:50.961078 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00a34a4-759a-4250-8b16-b19762b21d32 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:32:50.967248 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00a34a4-759a-4250-8b16-b19762b21d32 service placement] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:32:50.972858 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1210/2422] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:32:50 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:33:01.365017 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2aabd12-df8a-403c-915c-da3faa2a3f95 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:33:01.369638 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-a2aabd12-df8a-403c-915c-da3faa2a3f95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:33:01.462904 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2aabd12-df8a-403c-915c-da3faa2a3f95 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:33:01.467779 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac411376-0334-4119-a4b3-078497deabfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:33:01.468736 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac411376-0334-4119-a4b3-078497deabfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:33:01.469209 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac411376-0334-4119-a4b3-078497deabfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:33:01.469471 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac411376-0334-4119-a4b3-078497deabfe admin admin] PATH_INFO: `/v3/users/50392e3422f0473db858ed04379f193b` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:33:01.470889 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac411376-0334-4119-a4b3-078497deabfe admin admin] RBAC: Authorizing `identity:delete_user(user_id=50392e3422f0473db858ed04379f193b)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:33:01.473115 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac411376-0334-4119-a4b3-078497deabfe admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:33:01.703559 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-ac411376-0334-4119-a4b3-078497deabfe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50392e3422f0473db858ed04379f193b'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:33:01.711113 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-ac411376-0334-4119-a4b3-078497deabfe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50392e3422f0473db858ed04379f193b'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:33:01.733626 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-ac411376-0334-4119-a4b3-078497deabfe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50392e3422f0473db858ed04379f193b'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:33:01.734358 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-ac411376-0334-4119-a4b3-078497deabfe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50392e3422f0473db858ed04379f193b'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:33:01.735075 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-ac411376-0334-4119-a4b3-078497deabfe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50392e3422f0473db858ed04379f193b'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:33:01.802424 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1213/2423] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 17 19:33:01 2022] DELETE /identity/v3/users/50392e3422f0473db858ed04379f193b => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:33:01.806914 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e5d998-bbf2-48f6-8b29-2c36aeb0a488 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:33:01.807969 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e5d998-bbf2-48f6-8b29-2c36aeb0a488 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:33:01.807969 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e5d998-bbf2-48f6-8b29-2c36aeb0a488 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:33:01.826001 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-13e5d998-bbf2-48f6-8b29-2c36aeb0a488 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:33:01.827164 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-13e5d998-bbf2-48f6-8b29-2c36aeb0a488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:33:01.888643 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1211/2424] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:33:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:33:01.898167 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac411376-0334-4119-a4b3-078497deabfe admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:33:01.899736 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-ac411376-0334-4119-a4b3-078497deabfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:33:01.958706 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac411376-0334-4119-a4b3-078497deabfe admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:33:01.962480 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fbe3cf-cb05-4453-ad6e-05fa410ce60d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:33:01.963149 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fbe3cf-cb05-4453-ad6e-05fa410ce60d service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:33:01.963436 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fbe3cf-cb05-4453-ad6e-05fa410ce60d service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:33:01.963740 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fbe3cf-cb05-4453-ad6e-05fa410ce60d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:33:01.973165 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fbe3cf-cb05-4453-ad6e-05fa410ce60d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:33:01.974958 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fbe3cf-cb05-4453-ad6e-05fa410ce60d service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:33:01.984747 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1214/2425] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:33:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:33:02.231512 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f30ebef-49ff-4ef4-8bf4-0df3d99d8b02 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:33:02.231798 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f30ebef-49ff-4ef4-8bf4-0df3d99d8b02 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:33:02.232079 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f30ebef-49ff-4ef4-8bf4-0df3d99d8b02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:33:02.257286 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-4f30ebef-49ff-4ef4-8bf4-0df3d99d8b02 None None] MFA Rules not processed for user `2b90b57c542440448579b3aadda8b241`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:33:02.258690 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-4f30ebef-49ff-4ef4-8bf4-0df3d99d8b02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:33:02.327730 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1212/2426] 10.222.0.58 () {58 vars in 967 bytes} [Tue May 17 19:33:02 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:33:02.351453 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fbe3cf-cb05-4453-ad6e-05fa410ce60d service neutron] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:33:02.353142 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-15fbe3cf-cb05-4453-ad6e-05fa410ce60d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:33:02.418991 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f30ebef-49ff-4ef4-8bf4-0df3d99d8b02 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:33:02.420629 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-4f30ebef-49ff-4ef4-8bf4-0df3d99d8b02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:33:02.424437 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fbe3cf-cb05-4453-ad6e-05fa410ce60d service neutron] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:33:02.429796 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b204c29-c214-4070-8712-3321b4f59d71 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0cb292002434d30bc81f370d120a2a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:33:02.430447 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b204c29-c214-4070-8712-3321b4f59d71 service swift] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:33:02.430727 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b204c29-c214-4070-8712-3321b4f59d71 service swift] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:33:02.431001 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b204c29-c214-4070-8712-3321b4f59d71 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:33:02.439634 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b204c29-c214-4070-8712-3321b4f59d71 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:33:02.440126 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f30ebef-49ff-4ef4-8bf4-0df3d99d8b02 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:33:02.441252 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b204c29-c214-4070-8712-3321b4f59d71 service swift] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:33:02.446755 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6469417-453e-48f6-b1c6-6e7709fbfdac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:33:02.448548 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6469417-453e-48f6-b1c6-6e7709fbfdac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:33:02.448994 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6469417-453e-48f6-b1c6-6e7709fbfdac admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:33:02.449444 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6469417-453e-48f6-b1c6-6e7709fbfdac admin admin] PATH_INFO: `/v3/projects/43709e6b4496459e99b07296a3bea2e9` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:33:02.450303 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1215/2427] 10.222.0.58 () {60 vars in 1384 bytes} [Tue May 17 19:33:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 17 19:33:02.450828 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6469417-453e-48f6-b1c6-6e7709fbfdac admin admin] RBAC: Authorizing `identity:delete_project(project_id=43709e6b4496459e99b07296a3bea2e9)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:33:02.453549 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6469417-453e-48f6-b1c6-6e7709fbfdac admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:33:02.562304 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f6469417-453e-48f6-b1c6-6e7709fbfdac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43709e6b4496459e99b07296a3bea2e9', 'request_id': 'req-f6469417-453e-48f6-b1c6-6e7709fbfdac'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:33:02.567721 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f6469417-453e-48f6-b1c6-6e7709fbfdac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43709e6b4496459e99b07296a3bea2e9', 'request_id': 'req-f6469417-453e-48f6-b1c6-6e7709fbfdac'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:33:02.634084 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f6469417-453e-48f6-b1c6-6e7709fbfdac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43709e6b4496459e99b07296a3bea2e9', 'request_id': 'req-f6469417-453e-48f6-b1c6-6e7709fbfdac'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:33:02.646184 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f6469417-453e-48f6-b1c6-6e7709fbfdac admin admin] The token cache is being invalidate because project 43709e6b4496459e99b07296a3bea2e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 17 19:33:02.646601 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f6469417-453e-48f6-b1c6-6e7709fbfdac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:33:02.647256 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f6469417-453e-48f6-b1c6-6e7709fbfdac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:33:02.649129 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1213/2428] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 17 19:33:02 2022] DELETE /identity/v3/projects/43709e6b4496459e99b07296a3bea2e9 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 17 19:33:02.657161 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131fed43-11f7-4ccd-ba36-64ec1aa2a1dd None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:33:02.657445 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131fed43-11f7-4ccd-ba36-64ec1aa2a1dd None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:33:02.657887 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131fed43-11f7-4ccd-ba36-64ec1aa2a1dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:33:02.678208 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-131fed43-11f7-4ccd-ba36-64ec1aa2a1dd None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:33:02.679410 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-131fed43-11f7-4ccd-ba36-64ec1aa2a1dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:33:02.780898 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1216/2429] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:33:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:33:02.787359 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1fa500-960c-48f7-a8b0-cf1f6a073737 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:33:02.787726 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1fa500-960c-48f7-a8b0-cf1f6a073737 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:33:02.788179 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1fa500-960c-48f7-a8b0-cf1f6a073737 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:33:02.809851 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-ae1fa500-960c-48f7-a8b0-cf1f6a073737 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:33:02.811853 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-ae1fa500-960c-48f7-a8b0-cf1f6a073737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:33:02.821752 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1214/2430] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:33:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:33:02.826647 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131fed43-11f7-4ccd-ba36-64ec1aa2a1dd None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:33:02.837308 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131fed43-11f7-4ccd-ba36-64ec1aa2a1dd None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:33:02.843444 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebda6530-3ec1-4472-9bdd-2df4d43545ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:33:02.844174 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebda6530-3ec1-4472-9bdd-2df4d43545ef admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:33:02.844453 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebda6530-3ec1-4472-9bdd-2df4d43545ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:33:02.844913 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebda6530-3ec1-4472-9bdd-2df4d43545ef admin admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:33:02.845650 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebda6530-3ec1-4472-9bdd-2df4d43545ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:33:02.845961 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebda6530-3ec1-4472-9bdd-2df4d43545ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:33:02.847709 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebda6530-3ec1-4472-9bdd-2df4d43545ef admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:33:02.866806 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1217/2431] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 17 19:33:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:33:02.871887 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1fa500-960c-48f7-a8b0-cf1f6a073737 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:33:02.887602 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1fa500-960c-48f7-a8b0-cf1f6a073737 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:33:02.893782 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a4c8dc-2899-4db3-9ffa-3dca3bcfdf28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:33:02.894475 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a4c8dc-2899-4db3-9ffa-3dca3bcfdf28 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:33:02.894934 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a4c8dc-2899-4db3-9ffa-3dca3bcfdf28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:33:02.895204 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a4c8dc-2899-4db3-9ffa-3dca3bcfdf28 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:33:02.895974 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a4c8dc-2899-4db3-9ffa-3dca3bcfdf28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:33:02.897909 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a4c8dc-2899-4db3-9ffa-3dca3bcfdf28 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:33:03.052782 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1215/2432] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 17 19:33:02 2022] POST /identity/v3/projects => generated 373 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:33:03.056808 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebda6530-3ec1-4472-9bdd-2df4d43545ef admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:33:03.066896 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebda6530-3ec1-4472-9bdd-2df4d43545ef admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:33:03.122453 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75ab717-fb70-416d-884f-6a62b41bbe0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:33:03.123129 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75ab717-fb70-416d-884f-6a62b41bbe0c admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:33:03.123408 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75ab717-fb70-416d-884f-6a62b41bbe0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:33:03.123779 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75ab717-fb70-416d-884f-6a62b41bbe0c admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:33:03.124471 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75ab717-fb70-416d-884f-6a62b41bbe0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:33:03.126390 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75ab717-fb70-416d-884f-6a62b41bbe0c admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:33:03.243325 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1218/2433] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 17 19:33:03 2022] POST /identity/v3/users => generated 287 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:33:03.247807 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a4c8dc-2899-4db3-9ffa-3dca3bcfdf28 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:33:03.257196 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a4c8dc-2899-4db3-9ffa-3dca3bcfdf28 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:33:03.262242 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f066697-a385-4f83-a066-1a5ed611be15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:33:03.262884 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f066697-a385-4f83-a066-1a5ed611be15 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:33:03.263150 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f066697-a385-4f83-a066-1a5ed611be15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:33:03.263415 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f066697-a385-4f83-a066-1a5ed611be15 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:33:03.264155 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f066697-a385-4f83-a066-1a5ed611be15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:33:03.265728 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f066697-a385-4f83-a066-1a5ed611be15 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:33:03.275755 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1216/2434] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:33:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 17 19:33:03.280496 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75ab717-fb70-416d-884f-6a62b41bbe0c admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:33:03.290417 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75ab717-fb70-416d-884f-6a62b41bbe0c admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:33:03.294985 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691ef918-fb5c-4881-851c-1208e2d20004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:33:03.295674 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691ef918-fb5c-4881-851c-1208e2d20004 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:33:03.295956 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691ef918-fb5c-4881-851c-1208e2d20004 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:33:03.296232 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691ef918-fb5c-4881-851c-1208e2d20004 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:33:03.296946 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691ef918-fb5c-4881-851c-1208e2d20004 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:33:03.298572 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691ef918-fb5c-4881-851c-1208e2d20004 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:33:03.309184 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1219/2435] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:33:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:33:03.313135 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f066697-a385-4f83-a066-1a5ed611be15 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:33:03.323707 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f066697-a385-4f83-a066-1a5ed611be15 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:33:03.328419 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e170ff7-5021-43ca-9e96-afdd840a89f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:33:03.329287 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e170ff7-5021-43ca-9e96-afdd840a89f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:33:03.329566 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e170ff7-5021-43ca-9e96-afdd840a89f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:33:03.329954 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e170ff7-5021-43ca-9e96-afdd840a89f9 admin admin] PATH_INFO: `/v3/projects/6ce5e37117f9430bb8fa0cd2723d27e6/users/44ed90ebd3f6451ba89b17e0d25b414b/roles/114e6a16e63447d1924707ace5aace25` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:33:03.344705 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e170ff7-5021-43ca-9e96-afdd840a89f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ce5e37117f9430bb8fa0cd2723d27e6, user_id=44ed90ebd3f6451ba89b17e0d25b414b, role_id=114e6a16e63447d1924707ace5aace25)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:33:03.347394 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e170ff7-5021-43ca-9e96-afdd840a89f9 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:33:03.414355 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1217/2436] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:33:03 2022] PUT /identity/v3/projects/6ce5e37117f9430bb8fa0cd2723d27e6/users/44ed90ebd3f6451ba89b17e0d25b414b/roles/114e6a16e63447d1924707ace5aace25 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:33:03.425273 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34741be6-7d03-45f3-aea8-94be5aa57ae9 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:33:03.425871 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34741be6-7d03-45f3-aea8-94be5aa57ae9 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:33:03.425871 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34741be6-7d03-45f3-aea8-94be5aa57ae9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:33:03.453518 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-34741be6-7d03-45f3-aea8-94be5aa57ae9 None None] MFA Rules not processed for user `44ed90ebd3f6451ba89b17e0d25b414b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:33:03.454754 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-34741be6-7d03-45f3-aea8-94be5aa57ae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:33:03.524791 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1220/2437] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:33:03 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:33:03.536166 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e170ff7-5021-43ca-9e96-afdd840a89f9 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:33:03.537836 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-3e170ff7-5021-43ca-9e96-afdd840a89f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:33:03.615722 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e170ff7-5021-43ca-9e96-afdd840a89f9 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:33:03.619702 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1165e398-1e8f-41df-8774-2d5ee1c7ce51 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd05409b38ae346458abfe41ebda1c59e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:33:03.620366 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1165e398-1e8f-41df-8774-2d5ee1c7ce51 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:33:03.620641 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1165e398-1e8f-41df-8774-2d5ee1c7ce51 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:33:03.620910 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1165e398-1e8f-41df-8774-2d5ee1c7ce51 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:33:03.629862 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1165e398-1e8f-41df-8774-2d5ee1c7ce51 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:33:03.631316 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1165e398-1e8f-41df-8774-2d5ee1c7ce51 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:33:03.641184 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1218/2438] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:33:03 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:33:05.385152 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6941319e-e587-4fe8-9785-ed3665c1d38b None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:33:05.386165 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6941319e-e587-4fe8-9785-ed3665c1d38b None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:33:05.386260 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6941319e-e587-4fe8-9785-ed3665c1d38b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:33:05.403818 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-6941319e-e587-4fe8-9785-ed3665c1d38b None None] MFA Rules not processed for user `44ed90ebd3f6451ba89b17e0d25b414b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:33:05.405002 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-6941319e-e587-4fe8-9785-ed3665c1d38b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:33:05.411898 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1221/2439] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:33:05 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:33:15.460146 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cf6ca4-4039-4c70-a774-b5dea8014da9 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:33:15.460728 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cf6ca4-4039-4c70-a774-b5dea8014da9 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:33:15.460728 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cf6ca4-4039-4c70-a774-b5dea8014da9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:33:15.480900 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-51cf6ca4-4039-4c70-a774-b5dea8014da9 None None] MFA Rules not processed for user `af0c3a3e5a934b01aad4d7886bee66b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:33:15.482111 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-51cf6ca4-4039-4c70-a774-b5dea8014da9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:33:15.537202 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1219/2440] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:33:15 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:33:21.831836 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e8dc61-defe-45d6-a4bb-0b6a29c7208f None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:33:21.832353 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e8dc61-defe-45d6-a4bb-0b6a29c7208f None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:33:21.832468 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e8dc61-defe-45d6-a4bb-0b6a29c7208f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:33:21.850581 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-a6e8dc61-defe-45d6-a4bb-0b6a29c7208f None None] MFA Rules not processed for user `2b90b57c542440448579b3aadda8b241`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:33:21.854240 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-a6e8dc61-defe-45d6-a4bb-0b6a29c7208f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:33:21.909327 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1222/2441] 10.222.0.58 () {58 vars in 1009 bytes} [Tue May 17 19:33:21 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:33:21.922596 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cf6ca4-4039-4c70-a774-b5dea8014da9 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:33:21.924076 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-51cf6ca4-4039-4c70-a774-b5dea8014da9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:33:21.996059 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cf6ca4-4039-4c70-a774-b5dea8014da9 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:33:22.000976 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0ec6ee-379d-45da-9ea0-c6c1938ce989 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0cb292002434d30bc81f370d120a2a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:33:22.001820 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0ec6ee-379d-45da-9ea0-c6c1938ce989 service swift] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:33:22.002101 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0ec6ee-379d-45da-9ea0-c6c1938ce989 service swift] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:33:22.002371 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0ec6ee-379d-45da-9ea0-c6c1938ce989 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:33:22.010297 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0ec6ee-379d-45da-9ea0-c6c1938ce989 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:33:22.012417 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0ec6ee-379d-45da-9ea0-c6c1938ce989 service swift] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:33:22.020262 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1220/2442] 10.222.0.58 () {60 vars in 1384 bytes} [Tue May 17 19:33:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 17 19:33:23.496790 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e8dc61-defe-45d6-a4bb-0b6a29c7208f None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:33:23.499079 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-a6e8dc61-defe-45d6-a4bb-0b6a29c7208f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:33:23.577578 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e8dc61-defe-45d6-a4bb-0b6a29c7208f None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:33:23.582221 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695128e8-76ed-44b3-b94d-11917e8a99be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:33:23.583072 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695128e8-76ed-44b3-b94d-11917e8a99be service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:33:23.583563 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695128e8-76ed-44b3-b94d-11917e8a99be service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:33:23.584003 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695128e8-76ed-44b3-b94d-11917e8a99be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:33:23.585982 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-695128e8-76ed-44b3-b94d-11917e8a99be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:33:23.672866 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695128e8-76ed-44b3-b94d-11917e8a99be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:33:23.674433 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695128e8-76ed-44b3-b94d-11917e8a99be service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:33:23.704337 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1223/2443] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:33:23 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:33:31.833460 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4858c720-4641-4cfd-9127-083e36729313 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:33:31.834007 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4858c720-4641-4cfd-9127-083e36729313 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:33:31.834007 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4858c720-4641-4cfd-9127-083e36729313 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:33:31.875734 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-4858c720-4641-4cfd-9127-083e36729313 None None] MFA Rules not processed for user `2b90b57c542440448579b3aadda8b241`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:33:31.875734 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-4858c720-4641-4cfd-9127-083e36729313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:33:31.899500 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1221/2444] 10.222.0.58 () {58 vars in 1009 bytes} [Tue May 17 19:33:31 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:33:31.920644 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695128e8-76ed-44b3-b94d-11917e8a99be service neutron] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:33:31.930697 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695128e8-76ed-44b3-b94d-11917e8a99be service neutron] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:33:31.936183 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8302ae6b-d2cc-4a41-8637-c1b9b668e8ad service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0cb292002434d30bc81f370d120a2a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:33:31.936951 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8302ae6b-d2cc-4a41-8637-c1b9b668e8ad service swift] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:33:31.937320 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8302ae6b-d2cc-4a41-8637-c1b9b668e8ad service swift] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:33:31.937670 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8302ae6b-d2cc-4a41-8637-c1b9b668e8ad service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:33:31.946737 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8302ae6b-d2cc-4a41-8637-c1b9b668e8ad service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:33:31.948580 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8302ae6b-d2cc-4a41-8637-c1b9b668e8ad service swift] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:33:31.971967 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1224/2445] 10.222.0.58 () {60 vars in 1384 bytes} [Tue May 17 19:33:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 17 19:33:33.174433 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9991ce99-3c51-44c8-837b-fe5f3b21e7fe None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:33:33.175867 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9991ce99-3c51-44c8-837b-fe5f3b21e7fe None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:33:33.176525 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9991ce99-3c51-44c8-837b-fe5f3b21e7fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:33:33.202092 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-9991ce99-3c51-44c8-837b-fe5f3b21e7fe None None] MFA Rules not processed for user `2b90b57c542440448579b3aadda8b241`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:33:33.203431 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-9991ce99-3c51-44c8-837b-fe5f3b21e7fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:33:33.211992 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1222/2446] 10.222.0.58 () {58 vars in 967 bytes} [Tue May 17 19:33:33 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:33:33.238782 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8302ae6b-d2cc-4a41-8637-c1b9b668e8ad service swift] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:33:33.252660 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8302ae6b-d2cc-4a41-8637-c1b9b668e8ad service swift] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:33:33.258044 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9102b969-64f0-4d44-86c8-ea85c79ef0ab service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0cb292002434d30bc81f370d120a2a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:33:33.259000 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9102b969-64f0-4d44-86c8-ea85c79ef0ab service swift] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:33:33.259611 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9102b969-64f0-4d44-86c8-ea85c79ef0ab service swift] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:33:33.260218 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9102b969-64f0-4d44-86c8-ea85c79ef0ab service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:33:33.270003 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9102b969-64f0-4d44-86c8-ea85c79ef0ab service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:33:33.271664 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9102b969-64f0-4d44-86c8-ea85c79ef0ab service swift] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:33:33.281548 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1225/2447] 10.222.0.58 () {60 vars in 1384 bytes} [Tue May 17 19:33:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 17 19:33:33.957005 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9b7ab6-3682-4e73-aba9-2024ac87f051 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:33:33.957005 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9b7ab6-3682-4e73-aba9-2024ac87f051 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:33:33.957005 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9b7ab6-3682-4e73-aba9-2024ac87f051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:33:33.984432 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-4b9b7ab6-3682-4e73-aba9-2024ac87f051 None None] MFA Rules not processed for user `2b90b57c542440448579b3aadda8b241`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:33:33.986106 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-4b9b7ab6-3682-4e73-aba9-2024ac87f051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:33:34.000626 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1223/2448] 10.222.0.58 () {58 vars in 1009 bytes} [Tue May 17 19:33:33 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:33:34.025115 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9102b969-64f0-4d44-86c8-ea85c79ef0ab service swift] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:33:34.037469 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9102b969-64f0-4d44-86c8-ea85c79ef0ab service swift] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:33:34.044217 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2430079-0b18-4d11-bb0e-4b98dd10bfe9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0cb292002434d30bc81f370d120a2a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:33:34.045069 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2430079-0b18-4d11-bb0e-4b98dd10bfe9 service swift] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:33:34.045514 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2430079-0b18-4d11-bb0e-4b98dd10bfe9 service swift] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:33:34.046050 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2430079-0b18-4d11-bb0e-4b98dd10bfe9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:33:34.067957 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2430079-0b18-4d11-bb0e-4b98dd10bfe9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:33:34.069995 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2430079-0b18-4d11-bb0e-4b98dd10bfe9 service swift] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:33:34.082065 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1226/2449] 10.222.0.58 () {60 vars in 1384 bytes} [Tue May 17 19:33:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 17 19:33:34.750576 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66439fc2-0361-4f4f-948f-73978f4b1946 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:33:34.751076 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66439fc2-0361-4f4f-948f-73978f4b1946 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:33:34.751173 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66439fc2-0361-4f4f-948f-73978f4b1946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:33:34.781122 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-66439fc2-0361-4f4f-948f-73978f4b1946 None None] MFA Rules not processed for user `2b90b57c542440448579b3aadda8b241`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:33:34.784372 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-66439fc2-0361-4f4f-948f-73978f4b1946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:33:34.795410 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1224/2450] 10.222.0.58 () {58 vars in 967 bytes} [Tue May 17 19:33:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:33:34.835797 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2430079-0b18-4d11-bb0e-4b98dd10bfe9 service swift] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:33:34.835797 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2430079-0b18-4d11-bb0e-4b98dd10bfe9 service swift] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:33:34.835797 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f6a044-c799-4639-b7e5-8cb0aa54316f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0cb292002434d30bc81f370d120a2a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:33:34.835797 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f6a044-c799-4639-b7e5-8cb0aa54316f service swift] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:33:34.836702 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f6a044-c799-4639-b7e5-8cb0aa54316f service swift] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:33:34.836702 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f6a044-c799-4639-b7e5-8cb0aa54316f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:33:34.840089 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f6a044-c799-4639-b7e5-8cb0aa54316f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:33:34.841773 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f6a044-c799-4639-b7e5-8cb0aa54316f service swift] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:33:34.851189 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1227/2451] 10.222.0.58 () {60 vars in 1384 bytes} [Tue May 17 19:33:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 17 19:33:38.419062 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66439fc2-0361-4f4f-948f-73978f4b1946 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:33:38.499234 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66439fc2-0361-4f4f-948f-73978f4b1946 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:33:38.503888 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10956b8d-7edf-4c54-894d-42d9e77b7611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:33:38.504836 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10956b8d-7edf-4c54-894d-42d9e77b7611 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:33:38.505187 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10956b8d-7edf-4c54-894d-42d9e77b7611 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:33:38.505569 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10956b8d-7edf-4c54-894d-42d9e77b7611 admin admin] PATH_INFO: `/v3/users/44ed90ebd3f6451ba89b17e0d25b414b` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:33:38.506775 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10956b8d-7edf-4c54-894d-42d9e77b7611 admin admin] RBAC: Authorizing `identity:delete_user(user_id=44ed90ebd3f6451ba89b17e0d25b414b)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:33:38.508758 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10956b8d-7edf-4c54-894d-42d9e77b7611 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:33:38.880357 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-10956b8d-7edf-4c54-894d-42d9e77b7611 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44ed90ebd3f6451ba89b17e0d25b414b'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:33:38.881008 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-10956b8d-7edf-4c54-894d-42d9e77b7611 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44ed90ebd3f6451ba89b17e0d25b414b'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:33:38.899065 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-10956b8d-7edf-4c54-894d-42d9e77b7611 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44ed90ebd3f6451ba89b17e0d25b414b'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:33:38.991580 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-10956b8d-7edf-4c54-894d-42d9e77b7611 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44ed90ebd3f6451ba89b17e0d25b414b'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:33:39.001297 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-10956b8d-7edf-4c54-894d-42d9e77b7611 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44ed90ebd3f6451ba89b17e0d25b414b'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:33:39.008274 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1225/2452] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 17 19:33:38 2022] DELETE /identity/v3/users/44ed90ebd3f6451ba89b17e0d25b414b => generated 0 bytes in 590 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:33:39.014116 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16c6d5e-5083-40b7-85fe-0cec735a5f4a None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:33:39.016977 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16c6d5e-5083-40b7-85fe-0cec735a5f4a None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:33:39.016977 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16c6d5e-5083-40b7-85fe-0cec735a5f4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:33:39.036529 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-d16c6d5e-5083-40b7-85fe-0cec735a5f4a None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:33:39.037681 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-d16c6d5e-5083-40b7-85fe-0cec735a5f4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:33:39.111072 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1228/2453] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:33:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:33:39.121138 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10956b8d-7edf-4c54-894d-42d9e77b7611 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:33:39.122578 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-10956b8d-7edf-4c54-894d-42d9e77b7611 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:33:39.189099 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10956b8d-7edf-4c54-894d-42d9e77b7611 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:33:39.193255 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc25c22-58a6-4eca-8ef2-2bb82bb95624 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:33:39.194012 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc25c22-58a6-4eca-8ef2-2bb82bb95624 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:33:39.194407 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc25c22-58a6-4eca-8ef2-2bb82bb95624 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:33:39.194786 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc25c22-58a6-4eca-8ef2-2bb82bb95624 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:33:39.204813 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc25c22-58a6-4eca-8ef2-2bb82bb95624 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:33:39.206432 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc25c22-58a6-4eca-8ef2-2bb82bb95624 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:33:39.224437 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1226/2454] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:33:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:33:39.497286 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16c6d5e-5083-40b7-85fe-0cec735a5f4a None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:33:39.500705 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-d16c6d5e-5083-40b7-85fe-0cec735a5f4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:33:39.525541 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16c6d5e-5083-40b7-85fe-0cec735a5f4a None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:33:39.532693 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e90434c-7316-4b88-a7fa-d90903483456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:33:39.533667 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e90434c-7316-4b88-a7fa-d90903483456 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:33:39.534008 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e90434c-7316-4b88-a7fa-d90903483456 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:33:39.537670 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e90434c-7316-4b88-a7fa-d90903483456 admin admin] PATH_INFO: `/v3/projects/6ce5e37117f9430bb8fa0cd2723d27e6` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:33:39.537670 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e90434c-7316-4b88-a7fa-d90903483456 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ce5e37117f9430bb8fa0cd2723d27e6)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:33:39.538404 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e90434c-7316-4b88-a7fa-d90903483456 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:33:39.673083 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-4e90434c-7316-4b88-a7fa-d90903483456 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ce5e37117f9430bb8fa0cd2723d27e6', 'request_id': 'req-4e90434c-7316-4b88-a7fa-d90903483456'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:33:39.679065 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-4e90434c-7316-4b88-a7fa-d90903483456 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ce5e37117f9430bb8fa0cd2723d27e6', 'request_id': 'req-4e90434c-7316-4b88-a7fa-d90903483456'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:33:39.684591 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-4e90434c-7316-4b88-a7fa-d90903483456 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ce5e37117f9430bb8fa0cd2723d27e6', 'request_id': 'req-4e90434c-7316-4b88-a7fa-d90903483456'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:33:39.787238 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-4e90434c-7316-4b88-a7fa-d90903483456 admin admin] The token cache is being invalidate because project 6ce5e37117f9430bb8fa0cd2723d27e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 17 19:33:39.787555 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-4e90434c-7316-4b88-a7fa-d90903483456 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:33:39.788170 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-4e90434c-7316-4b88-a7fa-d90903483456 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:33:39.789709 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1229/2455] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 17 19:33:39 2022] DELETE /identity/v3/projects/6ce5e37117f9430bb8fa0cd2723d27e6 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:33:40.930523 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf88e1b8-9a51-4c49-8c8a-52e67891c3a9 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:33:40.931033 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf88e1b8-9a51-4c49-8c8a-52e67891c3a9 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:33:40.931130 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf88e1b8-9a51-4c49-8c8a-52e67891c3a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:33:40.949212 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-cf88e1b8-9a51-4c49-8c8a-52e67891c3a9 None None] MFA Rules not processed for user `2b90b57c542440448579b3aadda8b241`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:33:40.950418 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-cf88e1b8-9a51-4c49-8c8a-52e67891c3a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:33:41.002995 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1227/2456] 10.222.0.58 () {58 vars in 967 bytes} [Tue May 17 19:33:40 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:33:41.016281 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e90434c-7316-4b88-a7fa-d90903483456 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:33:41.017672 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-4e90434c-7316-4b88-a7fa-d90903483456 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:33:41.083541 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e90434c-7316-4b88-a7fa-d90903483456 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:33:41.088524 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9ca624-d13a-4ef2-810b-ad6971a7bc27 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0cb292002434d30bc81f370d120a2a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:33:41.089170 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9ca624-d13a-4ef2-810b-ad6971a7bc27 service swift] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:33:41.089451 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9ca624-d13a-4ef2-810b-ad6971a7bc27 service swift] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:33:41.089726 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9ca624-d13a-4ef2-810b-ad6971a7bc27 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:33:41.097521 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9ca624-d13a-4ef2-810b-ad6971a7bc27 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:33:41.099093 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9ca624-d13a-4ef2-810b-ad6971a7bc27 service swift] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:33:41.107135 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1230/2457] 10.222.0.58 () {60 vars in 1384 bytes} [Tue May 17 19:33:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 17 19:33:44.288714 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf88e1b8-9a51-4c49-8c8a-52e67891c3a9 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:33:44.291324 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-cf88e1b8-9a51-4c49-8c8a-52e67891c3a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:33:44.354575 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf88e1b8-9a51-4c49-8c8a-52e67891c3a9 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:33:44.359036 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafc6b3d-ae04-408e-a7fe-986f4f5410d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:33:44.360061 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafc6b3d-ae04-408e-a7fe-986f4f5410d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:33:44.360349 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafc6b3d-ae04-408e-a7fe-986f4f5410d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:33:44.360622 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafc6b3d-ae04-408e-a7fe-986f4f5410d1 admin admin] PATH_INFO: `/v3/users/34e5f89510c04ac78c6f3a94dddc23de` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:33:44.361799 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafc6b3d-ae04-408e-a7fe-986f4f5410d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=34e5f89510c04ac78c6f3a94dddc23de)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:33:44.363818 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafc6b3d-ae04-408e-a7fe-986f4f5410d1 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:33:44.572302 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-aafc6b3d-ae04-408e-a7fe-986f4f5410d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34e5f89510c04ac78c6f3a94dddc23de'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:33:44.573621 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-aafc6b3d-ae04-408e-a7fe-986f4f5410d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34e5f89510c04ac78c6f3a94dddc23de'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:33:44.586789 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-aafc6b3d-ae04-408e-a7fe-986f4f5410d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34e5f89510c04ac78c6f3a94dddc23de'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:33:44.644866 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-aafc6b3d-ae04-408e-a7fe-986f4f5410d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34e5f89510c04ac78c6f3a94dddc23de'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:33:44.652843 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-aafc6b3d-ae04-408e-a7fe-986f4f5410d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34e5f89510c04ac78c6f3a94dddc23de'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:33:44.658777 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1228/2458] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 17 19:33:44 2022] DELETE /identity/v3/users/34e5f89510c04ac78c6f3a94dddc23de => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:33:44.921938 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9ca624-d13a-4ef2-810b-ad6971a7bc27 service swift] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:33:44.934958 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9ca624-d13a-4ef2-810b-ad6971a7bc27 service swift] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:33:44.993689 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe51045-cf14-4e91-b3a5-27309d03bbdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:33:44.994788 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe51045-cf14-4e91-b3a5-27309d03bbdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:33:44.995677 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe51045-cf14-4e91-b3a5-27309d03bbdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:33:44.996054 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe51045-cf14-4e91-b3a5-27309d03bbdb admin admin] PATH_INFO: `/v3/projects/526f0c023217424d8ddde06706fff79c` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:33:44.997346 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe51045-cf14-4e91-b3a5-27309d03bbdb admin admin] RBAC: Authorizing `identity:delete_project(project_id=526f0c023217424d8ddde06706fff79c)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:33:44.999933 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe51045-cf14-4e91-b3a5-27309d03bbdb admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:33:45.217754 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-cfe51045-cf14-4e91-b3a5-27309d03bbdb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '526f0c023217424d8ddde06706fff79c', 'request_id': 'req-cfe51045-cf14-4e91-b3a5-27309d03bbdb'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:33:45.223657 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-cfe51045-cf14-4e91-b3a5-27309d03bbdb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '526f0c023217424d8ddde06706fff79c', 'request_id': 'req-cfe51045-cf14-4e91-b3a5-27309d03bbdb'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:33:45.229817 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-cfe51045-cf14-4e91-b3a5-27309d03bbdb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '526f0c023217424d8ddde06706fff79c', 'request_id': 'req-cfe51045-cf14-4e91-b3a5-27309d03bbdb'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:33:45.304415 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-cfe51045-cf14-4e91-b3a5-27309d03bbdb admin admin] The token cache is being invalidate because project 526f0c023217424d8ddde06706fff79c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 17 19:33:45.304740 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-cfe51045-cf14-4e91-b3a5-27309d03bbdb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:33:45.305373 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-cfe51045-cf14-4e91-b3a5-27309d03bbdb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:33:45.306914 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1231/2459] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 17 19:33:44 2022] DELETE /identity/v3/projects/526f0c023217424d8ddde06706fff79c => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:33:45.311729 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013ae301-75a8-498d-911d-d7a0eea4afe4 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:33:45.312230 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013ae301-75a8-498d-911d-d7a0eea4afe4 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:33:45.312391 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013ae301-75a8-498d-911d-d7a0eea4afe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:33:45.329906 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-013ae301-75a8-498d-911d-d7a0eea4afe4 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:33:45.331027 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-013ae301-75a8-498d-911d-d7a0eea4afe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:33:45.391196 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1229/2460] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:33:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:33:45.402371 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637a75fa-f0f4-434d-ad82-e0571774c818 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:33:45.402653 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637a75fa-f0f4-434d-ad82-e0571774c818 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:33:45.402928 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637a75fa-f0f4-434d-ad82-e0571774c818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:33:45.422191 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-637a75fa-f0f4-434d-ad82-e0571774c818 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:33:45.423382 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-637a75fa-f0f4-434d-ad82-e0571774c818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:33:45.431164 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1232/2461] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:33:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:33:45.435461 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013ae301-75a8-498d-911d-d7a0eea4afe4 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:33:45.444850 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013ae301-75a8-498d-911d-d7a0eea4afe4 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:33:45.449392 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25ab61e-52fa-4313-a0ea-04c12f7c89f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:33:45.450025 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25ab61e-52fa-4313-a0ea-04c12f7c89f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:33:45.450299 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25ab61e-52fa-4313-a0ea-04c12f7c89f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:33:45.450564 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25ab61e-52fa-4313-a0ea-04c12f7c89f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:33:45.451263 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25ab61e-52fa-4313-a0ea-04c12f7c89f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:33:45.451593 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25ab61e-52fa-4313-a0ea-04c12f7c89f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:33:45.453187 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25ab61e-52fa-4313-a0ea-04c12f7c89f9 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:33:45.464946 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1230/2462] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 17 19:33:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:33:45.468982 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637a75fa-f0f4-434d-ad82-e0571774c818 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:33:45.478446 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637a75fa-f0f4-434d-ad82-e0571774c818 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:33:45.483843 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4213ca01-0b95-4b37-8a56-e9b8bce3ec7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:33:45.484504 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4213ca01-0b95-4b37-8a56-e9b8bce3ec7f admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:33:45.484781 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4213ca01-0b95-4b37-8a56-e9b8bce3ec7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:33:45.485047 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4213ca01-0b95-4b37-8a56-e9b8bce3ec7f admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:33:45.485751 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4213ca01-0b95-4b37-8a56-e9b8bce3ec7f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:33:45.487654 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4213ca01-0b95-4b37-8a56-e9b8bce3ec7f admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:33:45.568621 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1233/2463] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 17 19:33:45 2022] POST /identity/v3/projects => generated 383 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 17 19:33:45.572597 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25ab61e-52fa-4313-a0ea-04c12f7c89f9 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:33:45.582276 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25ab61e-52fa-4313-a0ea-04c12f7c89f9 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:33:45.637018 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d7e81f-82a1-4893-82f9-8217e7a5adf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:33:45.637667 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d7e81f-82a1-4893-82f9-8217e7a5adf7 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:33:45.637943 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d7e81f-82a1-4893-82f9-8217e7a5adf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:33:45.638208 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d7e81f-82a1-4893-82f9-8217e7a5adf7 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:33:45.638883 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d7e81f-82a1-4893-82f9-8217e7a5adf7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:33:45.640816 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d7e81f-82a1-4893-82f9-8217e7a5adf7 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:33:45.744564 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1231/2464] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 17 19:33:45 2022] POST /identity/v3/users => generated 292 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 17 19:33:45.748542 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4213ca01-0b95-4b37-8a56-e9b8bce3ec7f admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:33:45.758844 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4213ca01-0b95-4b37-8a56-e9b8bce3ec7f admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:33:45.764298 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fddf7e-2232-41d6-b090-5d5da17db14c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:33:45.764979 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fddf7e-2232-41d6-b090-5d5da17db14c admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:33:45.776693 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fddf7e-2232-41d6-b090-5d5da17db14c admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:33:45.776693 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fddf7e-2232-41d6-b090-5d5da17db14c admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:33:45.776693 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fddf7e-2232-41d6-b090-5d5da17db14c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:33:45.776693 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fddf7e-2232-41d6-b090-5d5da17db14c admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:33:45.778587 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1234/2465] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:33:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:33:45.782348 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d7e81f-82a1-4893-82f9-8217e7a5adf7 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:33:45.792141 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d7e81f-82a1-4893-82f9-8217e7a5adf7 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:33:45.796698 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67eb940b-bfa5-41e8-a6b1-51600ec9c6b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:33:45.797344 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eb940b-bfa5-41e8-a6b1-51600ec9c6b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:33:45.797622 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eb940b-bfa5-41e8-a6b1-51600ec9c6b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:33:45.799665 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eb940b-bfa5-41e8-a6b1-51600ec9c6b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:33:45.799665 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67eb940b-bfa5-41e8-a6b1-51600ec9c6b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:33:45.800246 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67eb940b-bfa5-41e8-a6b1-51600ec9c6b8 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:33:45.810176 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1232/2466] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:33:45 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:33:45.813913 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fddf7e-2232-41d6-b090-5d5da17db14c admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:33:45.824065 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fddf7e-2232-41d6-b090-5d5da17db14c admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:33:45.828558 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60024a84-a950-4d7c-8b89-fef61752ae16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:33:45.829457 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60024a84-a950-4d7c-8b89-fef61752ae16 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:33:45.829728 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60024a84-a950-4d7c-8b89-fef61752ae16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:33:45.829992 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60024a84-a950-4d7c-8b89-fef61752ae16 admin admin] PATH_INFO: `/v3/projects/0eb5668b412f42ce8932a0a90cd8c0e0/users/c7f90d829c914d89a95ff543ea0d12bd/roles/114e6a16e63447d1924707ace5aace25` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:33:45.844846 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60024a84-a950-4d7c-8b89-fef61752ae16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0eb5668b412f42ce8932a0a90cd8c0e0, user_id=c7f90d829c914d89a95ff543ea0d12bd, role_id=114e6a16e63447d1924707ace5aace25)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:33:45.847564 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60024a84-a950-4d7c-8b89-fef61752ae16 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:33:45.952030 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1235/2467] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:33:45 2022] PUT /identity/v3/projects/0eb5668b412f42ce8932a0a90cd8c0e0/users/c7f90d829c914d89a95ff543ea0d12bd/roles/114e6a16e63447d1924707ace5aace25 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:33:45.961965 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaf4ee3-f5ff-4155-b54b-78705c4afdcc None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:33:45.962254 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaf4ee3-f5ff-4155-b54b-78705c4afdcc None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:33:45.962527 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaf4ee3-f5ff-4155-b54b-78705c4afdcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:33:45.989302 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-dcaf4ee3-f5ff-4155-b54b-78705c4afdcc None None] MFA Rules not processed for user `c7f90d829c914d89a95ff543ea0d12bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:33:45.990503 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-dcaf4ee3-f5ff-4155-b54b-78705c4afdcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:33:46.052100 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1233/2468] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:33:45 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:33:46.063337 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60024a84-a950-4d7c-8b89-fef61752ae16 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:33:46.065051 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-60024a84-a950-4d7c-8b89-fef61752ae16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:33:46.118586 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60024a84-a950-4d7c-8b89-fef61752ae16 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:33:46.122738 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a11697-5c96-4c89-99a4-94f806d22176 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd05409b38ae346458abfe41ebda1c59e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:33:46.123364 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a11697-5c96-4c89-99a4-94f806d22176 service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:33:46.123795 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a11697-5c96-4c89-99a4-94f806d22176 service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:33:46.124072 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a11697-5c96-4c89-99a4-94f806d22176 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:33:46.132172 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a11697-5c96-4c89-99a4-94f806d22176 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:33:46.133582 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a11697-5c96-4c89-99a4-94f806d22176 service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:33:46.142642 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1236/2469] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:33:46 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:33:50.259832 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaf4ee3-f5ff-4155-b54b-78705c4afdcc None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:33:50.319247 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaf4ee3-f5ff-4155-b54b-78705c4afdcc None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:33:50.324224 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21269b93-9258-4892-b49a-061cf4666a04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:33:50.325106 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21269b93-9258-4892-b49a-061cf4666a04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:33:50.325380 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21269b93-9258-4892-b49a-061cf4666a04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:33:50.325650 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21269b93-9258-4892-b49a-061cf4666a04 admin admin] PATH_INFO: `/v3/users/c7f90d829c914d89a95ff543ea0d12bd` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:33:50.326815 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21269b93-9258-4892-b49a-061cf4666a04 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7f90d829c914d89a95ff543ea0d12bd)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:33:50.328784 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21269b93-9258-4892-b49a-061cf4666a04 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:33:50.542861 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-21269b93-9258-4892-b49a-061cf4666a04 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7f90d829c914d89a95ff543ea0d12bd'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:33:50.543540 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-21269b93-9258-4892-b49a-061cf4666a04 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7f90d829c914d89a95ff543ea0d12bd'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:33:50.554689 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-21269b93-9258-4892-b49a-061cf4666a04 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7f90d829c914d89a95ff543ea0d12bd'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:33:50.616016 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-21269b93-9258-4892-b49a-061cf4666a04 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7f90d829c914d89a95ff543ea0d12bd'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:33:50.623258 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-21269b93-9258-4892-b49a-061cf4666a04 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7f90d829c914d89a95ff543ea0d12bd'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:33:50.629093 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1234/2470] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 17 19:33:50 2022] DELETE /identity/v3/users/c7f90d829c914d89a95ff543ea0d12bd => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:33:50.633525 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2862589c-1d08-4e8a-8d1d-f8ee68f7d2a7 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:33:50.633978 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2862589c-1d08-4e8a-8d1d-f8ee68f7d2a7 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:33:50.634070 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2862589c-1d08-4e8a-8d1d-f8ee68f7d2a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:33:50.653545 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-2862589c-1d08-4e8a-8d1d-f8ee68f7d2a7 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:33:50.654793 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-2862589c-1d08-4e8a-8d1d-f8ee68f7d2a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:33:50.717302 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1237/2471] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:33:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:33:50.727255 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21269b93-9258-4892-b49a-061cf4666a04 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:33:50.728811 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-21269b93-9258-4892-b49a-061cf4666a04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:33:50.787871 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21269b93-9258-4892-b49a-061cf4666a04 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:33:50.791631 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efa7b16-8c7a-429f-a7be-20bba608c119 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:33:50.792268 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efa7b16-8c7a-429f-a7be-20bba608c119 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:33:50.792548 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efa7b16-8c7a-429f-a7be-20bba608c119 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:33:50.792822 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efa7b16-8c7a-429f-a7be-20bba608c119 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:33:50.801763 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efa7b16-8c7a-429f-a7be-20bba608c119 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:33:50.803243 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efa7b16-8c7a-429f-a7be-20bba608c119 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:33:50.813318 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1235/2472] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:33:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:33:51.278293 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2862589c-1d08-4e8a-8d1d-f8ee68f7d2a7 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:33:51.279729 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-2862589c-1d08-4e8a-8d1d-f8ee68f7d2a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:33:51.292528 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2862589c-1d08-4e8a-8d1d-f8ee68f7d2a7 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:33:51.297275 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703dbf23-163e-4b02-9c0f-2589d7397ebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:33:51.298180 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703dbf23-163e-4b02-9c0f-2589d7397ebe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:33:51.298563 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703dbf23-163e-4b02-9c0f-2589d7397ebe admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:33:51.298844 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703dbf23-163e-4b02-9c0f-2589d7397ebe admin admin] PATH_INFO: `/v3/projects/0eb5668b412f42ce8932a0a90cd8c0e0` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:33:51.300776 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703dbf23-163e-4b02-9c0f-2589d7397ebe admin admin] RBAC: Authorizing `identity:delete_project(project_id=0eb5668b412f42ce8932a0a90cd8c0e0)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:33:51.302702 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703dbf23-163e-4b02-9c0f-2589d7397ebe admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:33:51.407077 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-703dbf23-163e-4b02-9c0f-2589d7397ebe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0eb5668b412f42ce8932a0a90cd8c0e0', 'request_id': 'req-703dbf23-163e-4b02-9c0f-2589d7397ebe'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:33:51.412990 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-703dbf23-163e-4b02-9c0f-2589d7397ebe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0eb5668b412f42ce8932a0a90cd8c0e0', 'request_id': 'req-703dbf23-163e-4b02-9c0f-2589d7397ebe'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:33:51.418897 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-703dbf23-163e-4b02-9c0f-2589d7397ebe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0eb5668b412f42ce8932a0a90cd8c0e0', 'request_id': 'req-703dbf23-163e-4b02-9c0f-2589d7397ebe'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:33:51.509610 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-703dbf23-163e-4b02-9c0f-2589d7397ebe admin admin] The token cache is being invalidate because project 0eb5668b412f42ce8932a0a90cd8c0e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 17 19:33:51.509897 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-703dbf23-163e-4b02-9c0f-2589d7397ebe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:33:51.510499 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-703dbf23-163e-4b02-9c0f-2589d7397ebe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:33:51.512066 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1238/2473] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 17 19:33:51 2022] DELETE /identity/v3/projects/0eb5668b412f42ce8932a0a90cd8c0e0 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:33:57.049697 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca0cc66-dc0f-4aa1-9171-92c4eda2e039 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:33:57.050227 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca0cc66-dc0f-4aa1-9171-92c4eda2e039 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:33:57.050227 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca0cc66-dc0f-4aa1-9171-92c4eda2e039 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:33:57.067168 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-5ca0cc66-dc0f-4aa1-9171-92c4eda2e039 None None] MFA Rules not processed for user `b171cf633e6e437fb509857b0a2962e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:33:57.068445 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-5ca0cc66-dc0f-4aa1-9171-92c4eda2e039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:33:57.127048 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1236/2474] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:33:57 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:34:09.113541 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a73646-bd01-4f35-af47-270c86404690 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:34:09.114042 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a73646-bd01-4f35-af47-270c86404690 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:34:09.114153 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a73646-bd01-4f35-af47-270c86404690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:34:09.132652 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-d5a73646-bd01-4f35-af47-270c86404690 None None] MFA Rules not processed for user `6545bb5d5eb24c199b5d89091b2c1545`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:34:09.133872 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-d5a73646-bd01-4f35-af47-270c86404690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:34:09.187127 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1239/2475] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:34:09 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:34:32.518289 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca0cc66-dc0f-4aa1-9171-92c4eda2e039 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:34:32.519814 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-5ca0cc66-dc0f-4aa1-9171-92c4eda2e039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:34:32.583360 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca0cc66-dc0f-4aa1-9171-92c4eda2e039 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:34:32.588002 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e971ea4e-5b1a-4699-adb1-9cc1aa69f2cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:34:32.588858 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e971ea4e-5b1a-4699-adb1-9cc1aa69f2cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:34:32.589135 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e971ea4e-5b1a-4699-adb1-9cc1aa69f2cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:34:32.589404 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e971ea4e-5b1a-4699-adb1-9cc1aa69f2cc admin admin] PATH_INFO: `/v3/users/af0c3a3e5a934b01aad4d7886bee66b8` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:34:32.590570 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e971ea4e-5b1a-4699-adb1-9cc1aa69f2cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=af0c3a3e5a934b01aad4d7886bee66b8)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:34:32.592531 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e971ea4e-5b1a-4699-adb1-9cc1aa69f2cc admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:34:32.754959 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-e971ea4e-5b1a-4699-adb1-9cc1aa69f2cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af0c3a3e5a934b01aad4d7886bee66b8'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:34:32.755640 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-e971ea4e-5b1a-4699-adb1-9cc1aa69f2cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af0c3a3e5a934b01aad4d7886bee66b8'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:34:32.767571 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-e971ea4e-5b1a-4699-adb1-9cc1aa69f2cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af0c3a3e5a934b01aad4d7886bee66b8'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:34:32.835576 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-e971ea4e-5b1a-4699-adb1-9cc1aa69f2cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af0c3a3e5a934b01aad4d7886bee66b8'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:34:32.842264 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-e971ea4e-5b1a-4699-adb1-9cc1aa69f2cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af0c3a3e5a934b01aad4d7886bee66b8'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:34:32.848077 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1237/2476] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 17 19:34:32 2022] DELETE /identity/v3/users/af0c3a3e5a934b01aad4d7886bee66b8 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:34:33.106547 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a73646-bd01-4f35-af47-270c86404690 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:34:33.117421 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a73646-bd01-4f35-af47-270c86404690 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:34:33.172912 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86338b0-12f0-407c-a2bc-12ab68689aa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:34:33.173755 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86338b0-12f0-407c-a2bc-12ab68689aa4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:34:33.174029 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86338b0-12f0-407c-a2bc-12ab68689aa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:34:33.174297 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86338b0-12f0-407c-a2bc-12ab68689aa4 admin admin] PATH_INFO: `/v3/projects/4c80a1d247d44fb9b467b7e312d7ddec` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:34:33.175527 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86338b0-12f0-407c-a2bc-12ab68689aa4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c80a1d247d44fb9b467b7e312d7ddec)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:34:33.177418 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86338b0-12f0-407c-a2bc-12ab68689aa4 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:34:33.280305 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-d86338b0-12f0-407c-a2bc-12ab68689aa4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c80a1d247d44fb9b467b7e312d7ddec', 'request_id': 'req-d86338b0-12f0-407c-a2bc-12ab68689aa4'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:34:33.286078 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-d86338b0-12f0-407c-a2bc-12ab68689aa4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c80a1d247d44fb9b467b7e312d7ddec', 'request_id': 'req-d86338b0-12f0-407c-a2bc-12ab68689aa4'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:34:33.291936 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-d86338b0-12f0-407c-a2bc-12ab68689aa4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c80a1d247d44fb9b467b7e312d7ddec', 'request_id': 'req-d86338b0-12f0-407c-a2bc-12ab68689aa4'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:34:33.392303 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-d86338b0-12f0-407c-a2bc-12ab68689aa4 admin admin] The token cache is being invalidate because project 4c80a1d247d44fb9b467b7e312d7ddec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 17 19:34:33.392515 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-d86338b0-12f0-407c-a2bc-12ab68689aa4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:34:33.393126 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-d86338b0-12f0-407c-a2bc-12ab68689aa4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:34:33.394671 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1240/2477] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 17 19:34:33 2022] DELETE /identity/v3/projects/4c80a1d247d44fb9b467b7e312d7ddec => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:34:41.461349 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e971ea4e-5b1a-4699-adb1-9cc1aa69f2cc admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:34:41.462958 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-e971ea4e-5b1a-4699-adb1-9cc1aa69f2cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:34:41.519306 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e971ea4e-5b1a-4699-adb1-9cc1aa69f2cc admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:34:41.566996 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca2ca12-6679-4315-b4f8-943c818ec0dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:34:41.567676 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca2ca12-6679-4315-b4f8-943c818ec0dd service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:34:41.567957 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca2ca12-6679-4315-b4f8-943c818ec0dd service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:34:41.568230 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca2ca12-6679-4315-b4f8-943c818ec0dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:34:41.569976 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-8ca2ca12-6679-4315-b4f8-943c818ec0dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:34:41.636794 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca2ca12-6679-4315-b4f8-943c818ec0dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:34:41.638218 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca2ca12-6679-4315-b4f8-943c818ec0dd service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:34:41.648757 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1238/2478] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:34:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:34:42.817281 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86338b0-12f0-407c-a2bc-12ab68689aa4 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:34:42.818740 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-d86338b0-12f0-407c-a2bc-12ab68689aa4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:34:42.831148 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86338b0-12f0-407c-a2bc-12ab68689aa4 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:34:42.834652 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5939e8-faf3-4d7e-b277-074cc394664e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:34:42.835272 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5939e8-faf3-4d7e-b277-074cc394664e service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:34:42.835583 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5939e8-faf3-4d7e-b277-074cc394664e service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:34:42.835855 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5939e8-faf3-4d7e-b277-074cc394664e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:34:42.838146 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-7c5939e8-faf3-4d7e-b277-074cc394664e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:34:42.850588 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5939e8-faf3-4d7e-b277-074cc394664e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:34:42.852318 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5939e8-faf3-4d7e-b277-074cc394664e service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:34:42.863075 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1241/2479] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:34:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:34:51.338102 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca2ca12-6679-4315-b4f8-943c818ec0dd service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:34:51.346510 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca2ca12-6679-4315-b4f8-943c818ec0dd service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:34:51.350126 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd701150-b1ec-47c6-8798-b3044ef28eec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:34:51.350798 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd701150-b1ec-47c6-8798-b3044ef28eec service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:34:51.351075 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd701150-b1ec-47c6-8798-b3044ef28eec service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:34:51.351348 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd701150-b1ec-47c6-8798-b3044ef28eec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:34:51.354153 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-fd701150-b1ec-47c6-8798-b3044ef28eec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:34:51.410085 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd701150-b1ec-47c6-8798-b3044ef28eec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:34:51.411574 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd701150-b1ec-47c6-8798-b3044ef28eec service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:34:51.421170 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1239/2480] 10.222.0.58 () {60 vars in 1396 bytes} [Tue May 17 19:34:51 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3260 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:35:00.287931 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5939e8-faf3-4d7e-b277-074cc394664e service neutron] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:35:00.289504 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-7c5939e8-faf3-4d7e-b277-074cc394664e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:35:00.303052 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5939e8-faf3-4d7e-b277-074cc394664e service neutron] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:35:00.309241 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fedcb43-5615-4116-87fb-bf74ed7c955e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c1f878b0dc4d63a4f5dcb12fac1dc1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:35:00.313708 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fedcb43-5615-4116-87fb-bf74ed7c955e service nova] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:35:00.313708 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fedcb43-5615-4116-87fb-bf74ed7c955e service nova] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:35:00.313708 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fedcb43-5615-4116-87fb-bf74ed7c955e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:35:00.313708 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-7fedcb43-5615-4116-87fb-bf74ed7c955e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:35:00.324346 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fedcb43-5615-4116-87fb-bf74ed7c955e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:35:00.325902 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fedcb43-5615-4116-87fb-bf74ed7c955e service nova] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:35:00.336064 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1242/2481] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:35:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:35:02.176071 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd701150-b1ec-47c6-8798-b3044ef28eec service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:35:02.177473 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-fd701150-b1ec-47c6-8798-b3044ef28eec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:35:02.178731 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fedcb43-5615-4116-87fb-bf74ed7c955e service nova] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:35:02.180100 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-7fedcb43-5615-4116-87fb-bf74ed7c955e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:35:02.191254 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fedcb43-5615-4116-87fb-bf74ed7c955e service nova] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:35:02.192458 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd701150-b1ec-47c6-8798-b3044ef28eec service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:35:02.194691 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2759e6-434a-4ff5-b4f8-aef64d594e33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:35:02.195315 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2759e6-434a-4ff5-b4f8-aef64d594e33 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:35:02.195624 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2759e6-434a-4ff5-b4f8-aef64d594e33 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:35:02.195898 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2759e6-434a-4ff5-b4f8-aef64d594e33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:35:02.198107 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-9e2759e6-434a-4ff5-b4f8-aef64d594e33 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:35:02.198397 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f795cade-29ea-4573-9a8d-0b5bf9d5efb2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c1f878b0dc4d63a4f5dcb12fac1dc1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:35:02.199083 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f795cade-29ea-4573-9a8d-0b5bf9d5efb2 service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:35:02.199296 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f795cade-29ea-4573-9a8d-0b5bf9d5efb2 service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:35:02.199606 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f795cade-29ea-4573-9a8d-0b5bf9d5efb2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:35:02.201256 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-f795cade-29ea-4573-9a8d-0b5bf9d5efb2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:35:02.257714 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f795cade-29ea-4573-9a8d-0b5bf9d5efb2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:35:02.258980 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2759e6-434a-4ff5-b4f8-aef64d594e33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:35:02.259366 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f795cade-29ea-4573-9a8d-0b5bf9d5efb2 service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:35:02.261622 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2759e6-434a-4ff5-b4f8-aef64d594e33 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:35:02.269303 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1240/2482] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:35:02 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:35:02.271524 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1243/2483] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:35:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 17 19:35:06.156949 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f795cade-29ea-4573-9a8d-0b5bf9d5efb2 service nova] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:35:06.158398 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-f795cade-29ea-4573-9a8d-0b5bf9d5efb2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:35:06.173787 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f795cade-29ea-4573-9a8d-0b5bf9d5efb2 service nova] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:35:06.178620 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63576463-aa2e-4a9c-bcba-19edd664ce26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:35:06.179506 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63576463-aa2e-4a9c-bcba-19edd664ce26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:35:06.179798 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63576463-aa2e-4a9c-bcba-19edd664ce26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:35:06.180073 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63576463-aa2e-4a9c-bcba-19edd664ce26 admin admin] PATH_INFO: `/v3/users/b171cf633e6e437fb509857b0a2962e2` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:35:06.181245 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63576463-aa2e-4a9c-bcba-19edd664ce26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b171cf633e6e437fb509857b0a2962e2)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:35:06.183187 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63576463-aa2e-4a9c-bcba-19edd664ce26 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:35:06.339256 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-63576463-aa2e-4a9c-bcba-19edd664ce26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b171cf633e6e437fb509857b0a2962e2'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:35:06.339930 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-63576463-aa2e-4a9c-bcba-19edd664ce26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b171cf633e6e437fb509857b0a2962e2'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:35:06.351107 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-63576463-aa2e-4a9c-bcba-19edd664ce26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b171cf633e6e437fb509857b0a2962e2'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:35:06.444650 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-63576463-aa2e-4a9c-bcba-19edd664ce26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b171cf633e6e437fb509857b0a2962e2'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:35:06.455220 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-63576463-aa2e-4a9c-bcba-19edd664ce26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b171cf633e6e437fb509857b0a2962e2'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:35:06.461402 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1241/2484] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 17 19:35:06 2022] DELETE /identity/v3/users/b171cf633e6e437fb509857b0a2962e2 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:35:06.711119 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2759e6-434a-4ff5-b4f8-aef64d594e33 service neutron] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:35:06.723414 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2759e6-434a-4ff5-b4f8-aef64d594e33 service neutron] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:35:06.781934 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db038944-9df9-404d-bb40-d4eadd186aaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:35:06.782789 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db038944-9df9-404d-bb40-d4eadd186aaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:35:06.783067 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db038944-9df9-404d-bb40-d4eadd186aaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:35:06.783366 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db038944-9df9-404d-bb40-d4eadd186aaa admin admin] PATH_INFO: `/v3/projects/5d488b25563d4373b5c980ab060bc822` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:35:06.784592 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db038944-9df9-404d-bb40-d4eadd186aaa admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d488b25563d4373b5c980ab060bc822)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:35:06.786489 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db038944-9df9-404d-bb40-d4eadd186aaa admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:35:06.901738 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-db038944-9df9-404d-bb40-d4eadd186aaa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d488b25563d4373b5c980ab060bc822', 'request_id': 'req-db038944-9df9-404d-bb40-d4eadd186aaa'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:35:06.907813 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-db038944-9df9-404d-bb40-d4eadd186aaa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d488b25563d4373b5c980ab060bc822', 'request_id': 'req-db038944-9df9-404d-bb40-d4eadd186aaa'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:35:06.913136 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-db038944-9df9-404d-bb40-d4eadd186aaa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d488b25563d4373b5c980ab060bc822', 'request_id': 'req-db038944-9df9-404d-bb40-d4eadd186aaa'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:35:06.983729 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-db038944-9df9-404d-bb40-d4eadd186aaa admin admin] The token cache is being invalidate because project 5d488b25563d4373b5c980ab060bc822 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 17 19:35:06.983957 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-db038944-9df9-404d-bb40-d4eadd186aaa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:35:06.984590 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-db038944-9df9-404d-bb40-d4eadd186aaa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:35:06.986156 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1244/2485] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 17 19:35:06 2022] DELETE /identity/v3/projects/5d488b25563d4373b5c980ab060bc822 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:35:06.991293 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900d8bc9-69fe-45a0-a5b3-f197300be3ee None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:35:06.991641 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900d8bc9-69fe-45a0-a5b3-f197300be3ee None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:35:06.991922 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900d8bc9-69fe-45a0-a5b3-f197300be3ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:35:07.009510 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-900d8bc9-69fe-45a0-a5b3-f197300be3ee None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:35:07.011239 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-900d8bc9-69fe-45a0-a5b3-f197300be3ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:35:07.067236 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1242/2486] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:35:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:35:07.075356 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a7ae1b-d564-4ea2-8c3e-0ca531d7c06d None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:35:07.075676 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a7ae1b-d564-4ea2-8c3e-0ca531d7c06d None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:35:07.075956 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a7ae1b-d564-4ea2-8c3e-0ca531d7c06d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:35:07.094213 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-a8a7ae1b-d564-4ea2-8c3e-0ca531d7c06d None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:35:07.095443 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-a8a7ae1b-d564-4ea2-8c3e-0ca531d7c06d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:35:07.104203 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1245/2487] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:35:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:35:07.108154 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900d8bc9-69fe-45a0-a5b3-f197300be3ee None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:35:07.117661 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900d8bc9-69fe-45a0-a5b3-f197300be3ee None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:35:07.122251 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acd9cee-a2cd-4f4a-b656-5f1d8b6bbbb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:35:07.122886 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acd9cee-a2cd-4f4a-b656-5f1d8b6bbbb5 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:35:07.123160 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acd9cee-a2cd-4f4a-b656-5f1d8b6bbbb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:35:07.123431 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acd9cee-a2cd-4f4a-b656-5f1d8b6bbbb5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:35:07.124172 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acd9cee-a2cd-4f4a-b656-5f1d8b6bbbb5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:35:07.124495 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acd9cee-a2cd-4f4a-b656-5f1d8b6bbbb5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:35:07.126087 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acd9cee-a2cd-4f4a-b656-5f1d8b6bbbb5 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:35:07.137140 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1243/2488] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 17 19:35:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:35:07.141108 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a7ae1b-d564-4ea2-8c3e-0ca531d7c06d None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:35:07.149925 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a7ae1b-d564-4ea2-8c3e-0ca531d7c06d None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:35:07.154980 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574c6e2d-8782-442b-aa16-aa5fd06e8fd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:35:07.155637 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574c6e2d-8782-442b-aa16-aa5fd06e8fd4 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:35:07.155916 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574c6e2d-8782-442b-aa16-aa5fd06e8fd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:35:07.156185 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574c6e2d-8782-442b-aa16-aa5fd06e8fd4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:35:07.156867 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574c6e2d-8782-442b-aa16-aa5fd06e8fd4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:35:07.158778 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574c6e2d-8782-442b-aa16-aa5fd06e8fd4 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:35:07.278142 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1246/2489] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 17 19:35:07 2022] POST /identity/v3/projects => generated 367 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:35:07.282307 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acd9cee-a2cd-4f4a-b656-5f1d8b6bbbb5 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:35:07.292586 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acd9cee-a2cd-4f4a-b656-5f1d8b6bbbb5 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:35:07.345216 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e832ddf-93d8-4bc1-881d-0d8edbe5fd65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:35:07.345868 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e832ddf-93d8-4bc1-881d-0d8edbe5fd65 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:35:07.346143 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e832ddf-93d8-4bc1-881d-0d8edbe5fd65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:35:07.346408 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e832ddf-93d8-4bc1-881d-0d8edbe5fd65 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:35:07.347073 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e832ddf-93d8-4bc1-881d-0d8edbe5fd65 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:35:07.349005 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e832ddf-93d8-4bc1-881d-0d8edbe5fd65 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:35:07.458863 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1244/2490] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 17 19:35:07 2022] POST /identity/v3/users => generated 284 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:35:07.463525 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574c6e2d-8782-442b-aa16-aa5fd06e8fd4 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:35:07.473811 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574c6e2d-8782-442b-aa16-aa5fd06e8fd4 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:35:07.478866 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5b6ecd-df5d-4f9d-abc4-38140da6f8d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:35:07.479566 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5b6ecd-df5d-4f9d-abc4-38140da6f8d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:35:07.479845 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5b6ecd-df5d-4f9d-abc4-38140da6f8d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:35:07.480116 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5b6ecd-df5d-4f9d-abc4-38140da6f8d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:35:07.480828 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5b6ecd-df5d-4f9d-abc4-38140da6f8d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:35:07.482426 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5b6ecd-df5d-4f9d-abc4-38140da6f8d2 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:35:07.492553 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1247/2491] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:35:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:35:07.496286 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e832ddf-93d8-4bc1-881d-0d8edbe5fd65 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:35:07.506340 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e832ddf-93d8-4bc1-881d-0d8edbe5fd65 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:35:07.511645 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac7c3e7-6942-4698-b623-8050db684373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:35:07.512286 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac7c3e7-6942-4698-b623-8050db684373 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:35:07.512563 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac7c3e7-6942-4698-b623-8050db684373 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:35:07.512834 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac7c3e7-6942-4698-b623-8050db684373 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:35:07.513538 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac7c3e7-6942-4698-b623-8050db684373 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:35:07.515229 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac7c3e7-6942-4698-b623-8050db684373 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:35:07.526294 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1245/2492] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:35:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:35:07.530541 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5b6ecd-df5d-4f9d-abc4-38140da6f8d2 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:35:07.540062 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5b6ecd-df5d-4f9d-abc4-38140da6f8d2 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:35:07.544727 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c082574-0f02-48ed-b6a4-0fa33694b3a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:35:07.545578 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c082574-0f02-48ed-b6a4-0fa33694b3a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:35:07.545857 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c082574-0f02-48ed-b6a4-0fa33694b3a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:35:07.546128 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c082574-0f02-48ed-b6a4-0fa33694b3a7 admin admin] PATH_INFO: `/v3/projects/ed18804f8943471fa78205bd7bd72bac/users/90dac8e18bed4d5abe12535cba4a670e/roles/114e6a16e63447d1924707ace5aace25` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:35:07.560066 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c082574-0f02-48ed-b6a4-0fa33694b3a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed18804f8943471fa78205bd7bd72bac, user_id=90dac8e18bed4d5abe12535cba4a670e, role_id=114e6a16e63447d1924707ace5aace25)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:35:07.562705 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c082574-0f02-48ed-b6a4-0fa33694b3a7 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:35:07.658398 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1248/2493] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:35:07 2022] PUT /identity/v3/projects/ed18804f8943471fa78205bd7bd72bac/users/90dac8e18bed4d5abe12535cba4a670e/roles/114e6a16e63447d1924707ace5aace25 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:35:07.668462 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724ed798-4e11-48f2-b4ff-eac31a21f123 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:35:07.668746 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724ed798-4e11-48f2-b4ff-eac31a21f123 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:35:07.669044 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724ed798-4e11-48f2-b4ff-eac31a21f123 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:35:07.697616 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-724ed798-4e11-48f2-b4ff-eac31a21f123 None None] MFA Rules not processed for user `90dac8e18bed4d5abe12535cba4a670e`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:35:07.698800 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-724ed798-4e11-48f2-b4ff-eac31a21f123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:35:07.758230 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1246/2494] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:35:07 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:35:07.763171 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c082574-0f02-48ed-b6a4-0fa33694b3a7 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:35:07.821543 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c082574-0f02-48ed-b6a4-0fa33694b3a7 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:35:07.826742 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df3b71f-8b3f-4602-99c3-fdaa91cba0f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:35:07.827383 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df3b71f-8b3f-4602-99c3-fdaa91cba0f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:35:07.827693 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df3b71f-8b3f-4602-99c3-fdaa91cba0f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:35:07.827967 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df3b71f-8b3f-4602-99c3-fdaa91cba0f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:35:07.828639 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df3b71f-8b3f-4602-99c3-fdaa91cba0f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:35:07.830525 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df3b71f-8b3f-4602-99c3-fdaa91cba0f1 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:35:07.949231 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1249/2495] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 17 19:35:07 2022] POST /identity/v3/projects => generated 371 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:35:07.953362 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724ed798-4e11-48f2-b4ff-eac31a21f123 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:35:07.963653 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724ed798-4e11-48f2-b4ff-eac31a21f123 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:35:08.020770 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37bca0c-3870-4e8b-905c-411608ab102c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:35:08.021436 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37bca0c-3870-4e8b-905c-411608ab102c admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:35:08.021741 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37bca0c-3870-4e8b-905c-411608ab102c admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:35:08.022010 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37bca0c-3870-4e8b-905c-411608ab102c admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:35:08.022694 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37bca0c-3870-4e8b-905c-411608ab102c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:35:08.024694 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37bca0c-3870-4e8b-905c-411608ab102c admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:35:08.103039 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1247/2496] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 17 19:35:07 2022] POST /identity/v3/users => generated 292 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:35:08.107506 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df3b71f-8b3f-4602-99c3-fdaa91cba0f1 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:35:08.117501 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df3b71f-8b3f-4602-99c3-fdaa91cba0f1 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:35:08.122787 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708a5d69-464a-4e2c-baf5-3e90e9a3a854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:35:08.123452 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708a5d69-464a-4e2c-baf5-3e90e9a3a854 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:35:08.123749 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708a5d69-464a-4e2c-baf5-3e90e9a3a854 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:35:08.124025 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708a5d69-464a-4e2c-baf5-3e90e9a3a854 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:35:08.124737 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708a5d69-464a-4e2c-baf5-3e90e9a3a854 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:35:08.126322 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708a5d69-464a-4e2c-baf5-3e90e9a3a854 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:35:08.136551 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1250/2497] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:35:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:35:08.140484 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37bca0c-3870-4e8b-905c-411608ab102c admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:35:08.150183 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37bca0c-3870-4e8b-905c-411608ab102c admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:35:08.154888 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cebee3-d2f3-4282-ae14-3d991279a567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:35:08.155791 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cebee3-d2f3-4282-ae14-3d991279a567 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:35:08.156069 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cebee3-d2f3-4282-ae14-3d991279a567 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:35:08.156343 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cebee3-d2f3-4282-ae14-3d991279a567 admin admin] PATH_INFO: `/v3/projects/b8b40caf39e44f8aa21c6ba36fea456a/users/7a858ebf6979449b8d844e833558d80a/roles/60af5088793e4379b88821a86552a344` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:35:08.170975 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cebee3-d2f3-4282-ae14-3d991279a567 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8b40caf39e44f8aa21c6ba36fea456a, user_id=7a858ebf6979449b8d844e833558d80a, role_id=60af5088793e4379b88821a86552a344)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:35:08.173662 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cebee3-d2f3-4282-ae14-3d991279a567 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:35:08.263688 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1248/2498] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:35:08 2022] PUT /identity/v3/projects/b8b40caf39e44f8aa21c6ba36fea456a/users/7a858ebf6979449b8d844e833558d80a/roles/60af5088793e4379b88821a86552a344 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:35:08.267536 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708a5d69-464a-4e2c-baf5-3e90e9a3a854 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:35:08.277427 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708a5d69-464a-4e2c-baf5-3e90e9a3a854 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:35:08.331138 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1bb80d-03ba-4207-80c8-dee2ca164cb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:35:08.331878 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1bb80d-03ba-4207-80c8-dee2ca164cb7 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:35:08.332158 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1bb80d-03ba-4207-80c8-dee2ca164cb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:35:08.332430 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1bb80d-03ba-4207-80c8-dee2ca164cb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:35:08.333145 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1bb80d-03ba-4207-80c8-dee2ca164cb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:35:08.334765 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1bb80d-03ba-4207-80c8-dee2ca164cb7 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:35:08.344197 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1251/2499] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:35:08 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:35:08.348194 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cebee3-d2f3-4282-ae14-3d991279a567 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:35:08.359044 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cebee3-d2f3-4282-ae14-3d991279a567 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:35:08.363466 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5452ab0d-98d9-4e0a-8f6e-966a36dc834d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:35:08.364385 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5452ab0d-98d9-4e0a-8f6e-966a36dc834d admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:35:08.364659 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5452ab0d-98d9-4e0a-8f6e-966a36dc834d admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:35:08.364932 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5452ab0d-98d9-4e0a-8f6e-966a36dc834d admin admin] PATH_INFO: `/v3/projects/b8b40caf39e44f8aa21c6ba36fea456a/users/7a858ebf6979449b8d844e833558d80a/roles/60af5088793e4379b88821a86552a344` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:35:08.366863 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5452ab0d-98d9-4e0a-8f6e-966a36dc834d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8b40caf39e44f8aa21c6ba36fea456a, user_id=7a858ebf6979449b8d844e833558d80a, role_id=60af5088793e4379b88821a86552a344)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:35:08.370157 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5452ab0d-98d9-4e0a-8f6e-966a36dc834d admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:35:08.379848 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1249/2500] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:35:08 2022] PUT /identity/v3/projects/b8b40caf39e44f8aa21c6ba36fea456a/users/7a858ebf6979449b8d844e833558d80a/roles/60af5088793e4379b88821a86552a344 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:35:08.390718 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd69a30-d86f-4523-a38f-b7e4237abe0c None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:35:08.391002 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd69a30-d86f-4523-a38f-b7e4237abe0c None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:35:08.391306 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd69a30-d86f-4523-a38f-b7e4237abe0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:35:08.417992 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-0bd69a30-d86f-4523-a38f-b7e4237abe0c None None] MFA Rules not processed for user `7a858ebf6979449b8d844e833558d80a`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:35:08.419186 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-0bd69a30-d86f-4523-a38f-b7e4237abe0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:35:08.483889 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1252/2501] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:35:08 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:35:08.496289 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5452ab0d-98d9-4e0a-8f6e-966a36dc834d admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:35:08.498298 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-5452ab0d-98d9-4e0a-8f6e-966a36dc834d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:35:08.553120 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5452ab0d-98d9-4e0a-8f6e-966a36dc834d admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:35:08.556715 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9978b4f6-4b41-48bd-8776-b9f2b3fefcd6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd05409b38ae346458abfe41ebda1c59e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:35:08.557345 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9978b4f6-4b41-48bd-8776-b9f2b3fefcd6 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:35:08.557623 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9978b4f6-4b41-48bd-8776-b9f2b3fefcd6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:35:08.557892 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9978b4f6-4b41-48bd-8776-b9f2b3fefcd6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:35:08.566596 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9978b4f6-4b41-48bd-8776-b9f2b3fefcd6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:35:08.568083 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9978b4f6-4b41-48bd-8776-b9f2b3fefcd6 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:35:08.577368 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1250/2502] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:35:08 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:35:08.822056 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd69a30-d86f-4523-a38f-b7e4237abe0c None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:35:08.831542 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd69a30-d86f-4523-a38f-b7e4237abe0c None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:35:08.835981 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d94b4e6-9df2-4b35-aec8-4269781b864d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd05409b38ae346458abfe41ebda1c59e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:35:08.835981 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d94b4e6-9df2-4b35-aec8-4269781b864d service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:35:08.836200 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d94b4e6-9df2-4b35-aec8-4269781b864d service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:35:08.836338 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d94b4e6-9df2-4b35-aec8-4269781b864d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:35:08.891503 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d94b4e6-9df2-4b35-aec8-4269781b864d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:35:08.892935 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d94b4e6-9df2-4b35-aec8-4269781b864d service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:35:08.902090 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1253/2503] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:35:08 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:35:19.076031 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e53204b-a1fa-4359-baab-fae914e0c45a None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:35:19.076561 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e53204b-a1fa-4359-baab-fae914e0c45a None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:35:19.076561 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e53204b-a1fa-4359-baab-fae914e0c45a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:35:19.094509 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-1e53204b-a1fa-4359-baab-fae914e0c45a None None] MFA Rules not processed for user `2b90b57c542440448579b3aadda8b241`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:35:19.095720 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-1e53204b-a1fa-4359-baab-fae914e0c45a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:35:19.148492 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1251/2504] 10.222.0.58 () {58 vars in 1009 bytes} [Tue May 17 19:35:19 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:35:19.282815 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d94b4e6-9df2-4b35-aec8-4269781b864d service cinder] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:35:19.284097 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-9d94b4e6-9df2-4b35-aec8-4269781b864d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:35:19.350412 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d94b4e6-9df2-4b35-aec8-4269781b864d service cinder] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:35:19.356231 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a852069-a66f-4256-a598-04d2822d569d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0cb292002434d30bc81f370d120a2a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:35:19.356868 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a852069-a66f-4256-a598-04d2822d569d service swift] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:35:19.357149 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a852069-a66f-4256-a598-04d2822d569d service swift] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:35:19.357424 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a852069-a66f-4256-a598-04d2822d569d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:35:19.365373 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a852069-a66f-4256-a598-04d2822d569d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:35:19.366944 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a852069-a66f-4256-a598-04d2822d569d service swift] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:35:19.375539 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1254/2505] 10.222.0.58 () {60 vars in 1384 bytes} [Tue May 17 19:35:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 17 19:35:20.507371 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e53204b-a1fa-4359-baab-fae914e0c45a None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:35:20.509219 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-1e53204b-a1fa-4359-baab-fae914e0c45a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:35:20.657419 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e53204b-a1fa-4359-baab-fae914e0c45a None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:35:20.665132 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff317ac-296b-4fe9-9ad9-7c1f7e5a48ac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c1f878b0dc4d63a4f5dcb12fac1dc1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:35:20.665816 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff317ac-296b-4fe9-9ad9-7c1f7e5a48ac service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:35:20.666148 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff317ac-296b-4fe9-9ad9-7c1f7e5a48ac service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:35:20.666424 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff317ac-296b-4fe9-9ad9-7c1f7e5a48ac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:35:20.668421 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-eff317ac-296b-4fe9-9ad9-7c1f7e5a48ac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:35:20.684366 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff317ac-296b-4fe9-9ad9-7c1f7e5a48ac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:35:20.686251 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff317ac-296b-4fe9-9ad9-7c1f7e5a48ac service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:35:20.698724 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1252/2506] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:35:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:35:30.652350 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a852069-a66f-4256-a598-04d2822d569d service swift] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:35:30.653731 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-2a852069-a66f-4256-a598-04d2822d569d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:35:30.707391 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a852069-a66f-4256-a598-04d2822d569d service swift] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:35:30.710937 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344a8541-16ff-4ef9-99ef-102f88ac934a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:35:30.711599 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344a8541-16ff-4ef9-99ef-102f88ac934a service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:35:30.711877 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344a8541-16ff-4ef9-99ef-102f88ac934a service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:35:30.712153 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344a8541-16ff-4ef9-99ef-102f88ac934a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:35:30.713860 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-344a8541-16ff-4ef9-99ef-102f88ac934a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:35:30.722733 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344a8541-16ff-4ef9-99ef-102f88ac934a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:35:30.724161 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344a8541-16ff-4ef9-99ef-102f88ac934a service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:35:30.732466 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1255/2507] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:35:30 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:35:37.436483 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff317ac-296b-4fe9-9ad9-7c1f7e5a48ac service nova] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:35:37.437882 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-eff317ac-296b-4fe9-9ad9-7c1f7e5a48ac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:35:37.504377 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff317ac-296b-4fe9-9ad9-7c1f7e5a48ac service nova] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:35:37.509399 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b60a5d-ea1a-434d-8453-a7d4df65805c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06538aab5f194af7b7c677d6b94aef2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:35:37.510043 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b60a5d-ea1a-434d-8453-a7d4df65805c service placement] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:35:37.510326 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b60a5d-ea1a-434d-8453-a7d4df65805c service placement] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:35:37.510605 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b60a5d-ea1a-434d-8453-a7d4df65805c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:35:37.512906 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-f2b60a5d-ea1a-434d-8453-a7d4df65805c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:35:37.524415 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b60a5d-ea1a-434d-8453-a7d4df65805c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:35:37.526002 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b60a5d-ea1a-434d-8453-a7d4df65805c service placement] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:35:37.536435 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1253/2508] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:35:37 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:35:42.430486 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e0438c-74b7-425d-9726-9e42bd84355a None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:35:42.431036 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e0438c-74b7-425d-9726-9e42bd84355a None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:35:42.431036 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e0438c-74b7-425d-9726-9e42bd84355a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:35:42.451452 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-86e0438c-74b7-425d-9726-9e42bd84355a None None] MFA Rules not processed for user `2b90b57c542440448579b3aadda8b241`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:35:42.454259 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-86e0438c-74b7-425d-9726-9e42bd84355a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:35:42.461695 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1256/2509] 10.222.0.58 () {58 vars in 1009 bytes} [Tue May 17 19:35:42 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:35:42.474365 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b60a5d-ea1a-434d-8453-a7d4df65805c service placement] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:35:42.485842 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b60a5d-ea1a-434d-8453-a7d4df65805c service placement] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:35:42.490782 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6207f612-b6fc-4018-9633-83937f4fe4f9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0cb292002434d30bc81f370d120a2a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:35:42.491418 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6207f612-b6fc-4018-9633-83937f4fe4f9 service swift] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:35:42.491724 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6207f612-b6fc-4018-9633-83937f4fe4f9 service swift] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:35:42.491999 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6207f612-b6fc-4018-9633-83937f4fe4f9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:35:42.500536 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6207f612-b6fc-4018-9633-83937f4fe4f9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:35:42.502834 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6207f612-b6fc-4018-9633-83937f4fe4f9 service swift] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:35:42.511899 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1254/2510] 10.222.0.58 () {60 vars in 1384 bytes} [Tue May 17 19:35:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 17 19:35:52.509304 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9860730c-3414-481c-b066-f7392e8b96da None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:35:52.509840 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9860730c-3414-481c-b066-f7392e8b96da None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:35:52.509840 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9860730c-3414-481c-b066-f7392e8b96da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:35:52.528349 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-9860730c-3414-481c-b066-f7392e8b96da None None] MFA Rules not processed for user `2e4f7fc1ab304418b892be060814bd8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:35:52.529536 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-9860730c-3414-481c-b066-f7392e8b96da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:35:52.588159 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1257/2511] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:35:52 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:35:52.899904 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6207f612-b6fc-4018-9633-83937f4fe4f9 service swift] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:35:52.901415 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-6207f612-b6fc-4018-9633-83937f4fe4f9 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:35:52.913039 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6207f612-b6fc-4018-9633-83937f4fe4f9 service swift] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:35:52.916540 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f32346-654b-441a-ac66-cadcadfa3bc6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:35:52.917236 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f32346-654b-441a-ac66-cadcadfa3bc6 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:35:52.917517 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f32346-654b-441a-ac66-cadcadfa3bc6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:35:52.917791 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f32346-654b-441a-ac66-cadcadfa3bc6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:35:52.919438 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b8f32346-654b-441a-ac66-cadcadfa3bc6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:35:52.928587 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f32346-654b-441a-ac66-cadcadfa3bc6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:35:52.930001 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f32346-654b-441a-ac66-cadcadfa3bc6 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:35:52.938371 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1255/2512] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:35:52 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:35:58.100453 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9860730c-3414-481c-b066-f7392e8b96da None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:35:58.116860 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9860730c-3414-481c-b066-f7392e8b96da None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:35:58.123252 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd59400f-1fa5-4eeb-a8e1-5c3cf7c4600a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c1f878b0dc4d63a4f5dcb12fac1dc1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:35:58.123576 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd59400f-1fa5-4eeb-a8e1-5c3cf7c4600a service nova] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:35:58.123855 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd59400f-1fa5-4eeb-a8e1-5c3cf7c4600a service nova] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:35:58.124125 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd59400f-1fa5-4eeb-a8e1-5c3cf7c4600a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:35:58.126700 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-fd59400f-1fa5-4eeb-a8e1-5c3cf7c4600a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:35:58.141143 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd59400f-1fa5-4eeb-a8e1-5c3cf7c4600a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:35:58.143250 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd59400f-1fa5-4eeb-a8e1-5c3cf7c4600a service nova] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:35:58.158497 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1258/2513] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:35:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:35:58.179732 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4528597-fb2a-47d3-a13a-1bddb18ccaa4 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:35:58.180635 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4528597-fb2a-47d3-a13a-1bddb18ccaa4 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:35:58.181131 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4528597-fb2a-47d3-a13a-1bddb18ccaa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:35:58.206089 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-d4528597-fb2a-47d3-a13a-1bddb18ccaa4 None None] MFA Rules not processed for user `2b90b57c542440448579b3aadda8b241`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:35:58.211386 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d4528597-fb2a-47d3-a13a-1bddb18ccaa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:35:58.221008 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1256/2514] 10.222.0.58 () {58 vars in 967 bytes} [Tue May 17 19:35:58 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 17 19:35:58.228483 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd59400f-1fa5-4eeb-a8e1-5c3cf7c4600a service nova] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:35:58.241444 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd59400f-1fa5-4eeb-a8e1-5c3cf7c4600a service nova] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:35:58.247543 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ef4954-d431-4a8a-b13f-3dea5c4d21bf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0cb292002434d30bc81f370d120a2a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:35:58.248528 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ef4954-d431-4a8a-b13f-3dea5c4d21bf service swift] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:35:58.248875 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ef4954-d431-4a8a-b13f-3dea5c4d21bf service swift] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:35:58.249342 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ef4954-d431-4a8a-b13f-3dea5c4d21bf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:35:58.259756 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ef4954-d431-4a8a-b13f-3dea5c4d21bf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:35:58.262939 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ef4954-d431-4a8a-b13f-3dea5c4d21bf service swift] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:35:58.271039 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1259/2515] 10.222.0.58 () {60 vars in 1384 bytes} [Tue May 17 19:35:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 17 19:35:59.430656 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4528597-fb2a-47d3-a13a-1bddb18ccaa4 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:35:59.499585 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4528597-fb2a-47d3-a13a-1bddb18ccaa4 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:35:59.503964 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd543b9d-3752-4cbb-842e-b9159e15beb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:35:59.504823 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd543b9d-3752-4cbb-842e-b9159e15beb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:35:59.505102 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd543b9d-3752-4cbb-842e-b9159e15beb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:35:59.505416 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd543b9d-3752-4cbb-842e-b9159e15beb7 admin admin] PATH_INFO: `/v3/users/90dac8e18bed4d5abe12535cba4a670e` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:35:59.506631 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd543b9d-3752-4cbb-842e-b9159e15beb7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=90dac8e18bed4d5abe12535cba4a670e)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:35:59.508586 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd543b9d-3752-4cbb-842e-b9159e15beb7 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:35:59.738871 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-cd543b9d-3752-4cbb-842e-b9159e15beb7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90dac8e18bed4d5abe12535cba4a670e'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:35:59.739548 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-cd543b9d-3752-4cbb-842e-b9159e15beb7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90dac8e18bed4d5abe12535cba4a670e'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:35:59.751959 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-cd543b9d-3752-4cbb-842e-b9159e15beb7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90dac8e18bed4d5abe12535cba4a670e'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:35:59.815931 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-cd543b9d-3752-4cbb-842e-b9159e15beb7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90dac8e18bed4d5abe12535cba4a670e'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:35:59.823163 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-cd543b9d-3752-4cbb-842e-b9159e15beb7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90dac8e18bed4d5abe12535cba4a670e'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:35:59.829643 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1257/2516] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 17 19:35:59 2022] DELETE /identity/v3/users/90dac8e18bed4d5abe12535cba4a670e => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:35:59.834262 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0640716-2030-4a0e-8cbc-0ff1c8a17533 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:35:59.834742 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0640716-2030-4a0e-8cbc-0ff1c8a17533 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:35:59.834742 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0640716-2030-4a0e-8cbc-0ff1c8a17533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:35:59.852432 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-c0640716-2030-4a0e-8cbc-0ff1c8a17533 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:35:59.854125 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-c0640716-2030-4a0e-8cbc-0ff1c8a17533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:35:59.911881 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1260/2517] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:35:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:35:59.921751 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd543b9d-3752-4cbb-842e-b9159e15beb7 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:35:59.923187 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-cd543b9d-3752-4cbb-842e-b9159e15beb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:35:59.980411 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd543b9d-3752-4cbb-842e-b9159e15beb7 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:35:59.984046 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d904e8-2be3-433d-9426-87c9898be313 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:35:59.984693 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d904e8-2be3-433d-9426-87c9898be313 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:35:59.984969 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d904e8-2be3-433d-9426-87c9898be313 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:35:59.985244 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d904e8-2be3-433d-9426-87c9898be313 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:35:59.994463 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d904e8-2be3-433d-9426-87c9898be313 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:35:59.996008 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d904e8-2be3-433d-9426-87c9898be313 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:00.006224 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1258/2518] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:35:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:36:00.379544 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0640716-2030-4a0e-8cbc-0ff1c8a17533 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:00.381033 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-c0640716-2030-4a0e-8cbc-0ff1c8a17533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:36:00.394225 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0640716-2030-4a0e-8cbc-0ff1c8a17533 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:00.399242 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd268eb-f2c0-497c-88ed-b56d76487273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:00.400128 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd268eb-f2c0-497c-88ed-b56d76487273 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:00.400407 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd268eb-f2c0-497c-88ed-b56d76487273 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:00.400677 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd268eb-f2c0-497c-88ed-b56d76487273 admin admin] PATH_INFO: `/v3/projects/ed18804f8943471fa78205bd7bd72bac` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:00.401928 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd268eb-f2c0-497c-88ed-b56d76487273 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed18804f8943471fa78205bd7bd72bac)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:00.403876 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd268eb-f2c0-497c-88ed-b56d76487273 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:00.515627 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-ebd268eb-f2c0-497c-88ed-b56d76487273 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed18804f8943471fa78205bd7bd72bac', 'request_id': 'req-ebd268eb-f2c0-497c-88ed-b56d76487273'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:00.521952 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-ebd268eb-f2c0-497c-88ed-b56d76487273 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed18804f8943471fa78205bd7bd72bac', 'request_id': 'req-ebd268eb-f2c0-497c-88ed-b56d76487273'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:00.528027 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-ebd268eb-f2c0-497c-88ed-b56d76487273 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed18804f8943471fa78205bd7bd72bac', 'request_id': 'req-ebd268eb-f2c0-497c-88ed-b56d76487273'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:00.624125 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-ebd268eb-f2c0-497c-88ed-b56d76487273 admin admin] The token cache is being invalidate because project ed18804f8943471fa78205bd7bd72bac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 17 19:36:00.624408 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-ebd268eb-f2c0-497c-88ed-b56d76487273 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:00.625047 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-ebd268eb-f2c0-497c-88ed-b56d76487273 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:00.626607 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1261/2519] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 17 19:36:00 2022] DELETE /identity/v3/projects/ed18804f8943471fa78205bd7bd72bac => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:36:00.630914 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d904e8-2be3-433d-9426-87c9898be313 service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:00.641166 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d904e8-2be3-433d-9426-87c9898be313 service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:00.699789 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec4b392-5032-49a6-a280-2dd9c9ba9492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:00.700712 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec4b392-5032-49a6-a280-2dd9c9ba9492 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:00.700989 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec4b392-5032-49a6-a280-2dd9c9ba9492 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:00.701257 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec4b392-5032-49a6-a280-2dd9c9ba9492 admin admin] PATH_INFO: `/v3/users/7a858ebf6979449b8d844e833558d80a` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:00.702422 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec4b392-5032-49a6-a280-2dd9c9ba9492 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a858ebf6979449b8d844e833558d80a)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:00.704375 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec4b392-5032-49a6-a280-2dd9c9ba9492 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:00.882599 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-8ec4b392-5032-49a6-a280-2dd9c9ba9492 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a858ebf6979449b8d844e833558d80a'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:00.883174 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-8ec4b392-5032-49a6-a280-2dd9c9ba9492 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a858ebf6979449b8d844e833558d80a'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:00.894422 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-8ec4b392-5032-49a6-a280-2dd9c9ba9492 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a858ebf6979449b8d844e833558d80a'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:00.964679 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-8ec4b392-5032-49a6-a280-2dd9c9ba9492 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a858ebf6979449b8d844e833558d80a'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:00.971787 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-8ec4b392-5032-49a6-a280-2dd9c9ba9492 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a858ebf6979449b8d844e833558d80a'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:00.977777 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1259/2520] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 17 19:36:00 2022] DELETE /identity/v3/users/7a858ebf6979449b8d844e833558d80a => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:36:01.385418 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd268eb-f2c0-497c-88ed-b56d76487273 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:01.387457 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-ebd268eb-f2c0-497c-88ed-b56d76487273 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:36:01.401167 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd268eb-f2c0-497c-88ed-b56d76487273 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:01.455873 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82281180-0aa8-42c6-b733-ce53b441a222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:01.456837 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82281180-0aa8-42c6-b733-ce53b441a222 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:01.457122 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82281180-0aa8-42c6-b733-ce53b441a222 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:01.457397 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82281180-0aa8-42c6-b733-ce53b441a222 admin admin] PATH_INFO: `/v3/projects/b8b40caf39e44f8aa21c6ba36fea456a` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:01.458571 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82281180-0aa8-42c6-b733-ce53b441a222 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8b40caf39e44f8aa21c6ba36fea456a)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:01.460520 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82281180-0aa8-42c6-b733-ce53b441a222 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:01.559941 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-82281180-0aa8-42c6-b733-ce53b441a222 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8b40caf39e44f8aa21c6ba36fea456a', 'request_id': 'req-82281180-0aa8-42c6-b733-ce53b441a222'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:01.565907 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-82281180-0aa8-42c6-b733-ce53b441a222 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8b40caf39e44f8aa21c6ba36fea456a', 'request_id': 'req-82281180-0aa8-42c6-b733-ce53b441a222'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:01.571395 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-82281180-0aa8-42c6-b733-ce53b441a222 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8b40caf39e44f8aa21c6ba36fea456a', 'request_id': 'req-82281180-0aa8-42c6-b733-ce53b441a222'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:01.645242 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-82281180-0aa8-42c6-b733-ce53b441a222 admin admin] The token cache is being invalidate because project b8b40caf39e44f8aa21c6ba36fea456a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 17 19:36:01.645457 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-82281180-0aa8-42c6-b733-ce53b441a222 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:01.646072 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-82281180-0aa8-42c6-b733-ce53b441a222 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:01.647771 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1262/2521] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 17 19:36:01 2022] DELETE /identity/v3/projects/b8b40caf39e44f8aa21c6ba36fea456a => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 17 19:36:01.653358 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0825e2a6-adbe-4de2-ab47-ec4b653c3326 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:01.653812 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0825e2a6-adbe-4de2-ab47-ec4b653c3326 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:01.653902 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0825e2a6-adbe-4de2-ab47-ec4b653c3326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:01.672515 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-0825e2a6-adbe-4de2-ab47-ec4b653c3326 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:36:01.673787 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-0825e2a6-adbe-4de2-ab47-ec4b653c3326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:36:01.732205 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1260/2522] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:36:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:36:01.739860 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a64f00-fb6a-4308-b032-f3198f277c7c None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:01.740150 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a64f00-fb6a-4308-b032-f3198f277c7c None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:01.740425 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a64f00-fb6a-4308-b032-f3198f277c7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:01.760004 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-08a64f00-fb6a-4308-b032-f3198f277c7c None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:36:01.761803 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-08a64f00-fb6a-4308-b032-f3198f277c7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:36:01.769650 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1263/2523] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:36:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:36:01.773927 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0825e2a6-adbe-4de2-ab47-ec4b653c3326 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:01.784004 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0825e2a6-adbe-4de2-ab47-ec4b653c3326 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:01.788554 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537ec097-c262-45e5-9990-a44113895048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:01.789200 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537ec097-c262-45e5-9990-a44113895048 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:01.789478 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537ec097-c262-45e5-9990-a44113895048 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:01.789752 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537ec097-c262-45e5-9990-a44113895048 admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:01.790469 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537ec097-c262-45e5-9990-a44113895048 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:36:01.790777 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537ec097-c262-45e5-9990-a44113895048 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:01.792963 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537ec097-c262-45e5-9990-a44113895048 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:01.804264 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1261/2524] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 17 19:36:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:36:01.808553 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a64f00-fb6a-4308-b032-f3198f277c7c None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:01.817526 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a64f00-fb6a-4308-b032-f3198f277c7c None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:01.822121 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89653e83-9c08-448a-b16d-62aba0fc7d00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:01.822760 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89653e83-9c08-448a-b16d-62aba0fc7d00 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:01.823035 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89653e83-9c08-448a-b16d-62aba0fc7d00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:01.823367 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89653e83-9c08-448a-b16d-62aba0fc7d00 admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:01.824098 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89653e83-9c08-448a-b16d-62aba0fc7d00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:01.825994 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89653e83-9c08-448a-b16d-62aba0fc7d00 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:01.916439 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1264/2525] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 17 19:36:01 2022] POST /identity/v3/projects => generated 363 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:36:01.921247 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537ec097-c262-45e5-9990-a44113895048 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:01.931045 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537ec097-c262-45e5-9990-a44113895048 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:01.985496 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14ab6e4-5444-4eac-83c3-aa6d0bafea79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:01.986157 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14ab6e4-5444-4eac-83c3-aa6d0bafea79 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:01.986436 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14ab6e4-5444-4eac-83c3-aa6d0bafea79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:01.986705 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14ab6e4-5444-4eac-83c3-aa6d0bafea79 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:01.987374 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14ab6e4-5444-4eac-83c3-aa6d0bafea79 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:01.989319 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14ab6e4-5444-4eac-83c3-aa6d0bafea79 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:02.055819 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1262/2526] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 17 19:36:01 2022] POST /identity/v3/users => generated 282 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 17 19:36:02.060118 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89653e83-9c08-448a-b16d-62aba0fc7d00 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:02.070396 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89653e83-9c08-448a-b16d-62aba0fc7d00 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:02.075721 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9f5889-e6ff-4e48-a352-5ea6ad17f888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:02.076393 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9f5889-e6ff-4e48-a352-5ea6ad17f888 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:02.076674 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9f5889-e6ff-4e48-a352-5ea6ad17f888 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:02.076944 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9f5889-e6ff-4e48-a352-5ea6ad17f888 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:02.077660 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9f5889-e6ff-4e48-a352-5ea6ad17f888 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:02.079250 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9f5889-e6ff-4e48-a352-5ea6ad17f888 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:02.089720 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1265/2527] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:36:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:36:02.093493 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14ab6e4-5444-4eac-83c3-aa6d0bafea79 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:02.103125 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14ab6e4-5444-4eac-83c3-aa6d0bafea79 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:02.107984 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed738493-731f-465e-9902-f14696844b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:02.108625 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed738493-731f-465e-9902-f14696844b35 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:02.108908 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed738493-731f-465e-9902-f14696844b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:02.109181 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed738493-731f-465e-9902-f14696844b35 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:02.109890 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed738493-731f-465e-9902-f14696844b35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:02.111478 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed738493-731f-465e-9902-f14696844b35 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:02.121105 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1263/2528] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:36:02 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:36:02.125561 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9f5889-e6ff-4e48-a352-5ea6ad17f888 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:02.134767 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9f5889-e6ff-4e48-a352-5ea6ad17f888 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:02.139891 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a1b485-4168-4bbc-bbb7-34a80ee52ee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:02.140761 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a1b485-4168-4bbc-bbb7-34a80ee52ee0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:02.141032 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a1b485-4168-4bbc-bbb7-34a80ee52ee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:02.141296 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a1b485-4168-4bbc-bbb7-34a80ee52ee0 admin admin] PATH_INFO: `/v3/projects/e318052251794abe9cbf360278e40967/users/3a6c13fea20e4cd98e77043a12ecdecf/roles/114e6a16e63447d1924707ace5aace25` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:02.155658 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a1b485-4168-4bbc-bbb7-34a80ee52ee0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e318052251794abe9cbf360278e40967, user_id=3a6c13fea20e4cd98e77043a12ecdecf, role_id=114e6a16e63447d1924707ace5aace25)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:02.158277 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a1b485-4168-4bbc-bbb7-34a80ee52ee0 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:02.221262 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1266/2529] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:36:02 2022] PUT /identity/v3/projects/e318052251794abe9cbf360278e40967/users/3a6c13fea20e4cd98e77043a12ecdecf/roles/114e6a16e63447d1924707ace5aace25 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 17 19:36:02.232323 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8ee797-fa39-4266-add7-8785cbc761a5 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:02.232607 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8ee797-fa39-4266-add7-8785cbc761a5 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:02.232882 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8ee797-fa39-4266-add7-8785cbc761a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:02.257939 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-0a8ee797-fa39-4266-add7-8785cbc761a5 None None] MFA Rules not processed for user `3a6c13fea20e4cd98e77043a12ecdecf`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:36:02.259114 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-0a8ee797-fa39-4266-add7-8785cbc761a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:36:02.318794 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1264/2530] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:36:02 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 17 19:36:02.323108 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a1b485-4168-4bbc-bbb7-34a80ee52ee0 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:02.383175 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a1b485-4168-4bbc-bbb7-34a80ee52ee0 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:02.388445 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0b1d1a-a7e3-463c-8cd6-b44ec22b6ec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:02.389087 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0b1d1a-a7e3-463c-8cd6-b44ec22b6ec0 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:02.389361 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0b1d1a-a7e3-463c-8cd6-b44ec22b6ec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:02.389627 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0b1d1a-a7e3-463c-8cd6-b44ec22b6ec0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:02.390295 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0b1d1a-a7e3-463c-8cd6-b44ec22b6ec0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:02.392218 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0b1d1a-a7e3-463c-8cd6-b44ec22b6ec0 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:02.469807 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1267/2531] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 17 19:36:02 2022] POST /identity/v3/projects => generated 363 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:36:02.473942 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8ee797-fa39-4266-add7-8785cbc761a5 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:02.485021 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8ee797-fa39-4266-add7-8785cbc761a5 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:02.544547 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094d0dcb-f7ea-40c7-a9b2-b557c213169b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:02.545218 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094d0dcb-f7ea-40c7-a9b2-b557c213169b admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:02.545494 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094d0dcb-f7ea-40c7-a9b2-b557c213169b admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:02.545761 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094d0dcb-f7ea-40c7-a9b2-b557c213169b admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:02.546442 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094d0dcb-f7ea-40c7-a9b2-b557c213169b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:02.548411 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094d0dcb-f7ea-40c7-a9b2-b557c213169b admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:02.628085 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1265/2532] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 17 19:36:02 2022] POST /identity/v3/users => generated 288 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:36:02.631899 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0b1d1a-a7e3-463c-8cd6-b44ec22b6ec0 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:02.642657 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0b1d1a-a7e3-463c-8cd6-b44ec22b6ec0 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:02.647303 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60a2f53-8ac5-47d1-907d-dd6722800905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:02.647889 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60a2f53-8ac5-47d1-907d-dd6722800905 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:02.648164 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60a2f53-8ac5-47d1-907d-dd6722800905 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:02.648433 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60a2f53-8ac5-47d1-907d-dd6722800905 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:02.649147 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60a2f53-8ac5-47d1-907d-dd6722800905 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:02.650730 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60a2f53-8ac5-47d1-907d-dd6722800905 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:02.660928 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1268/2533] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:36:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:36:02.665058 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094d0dcb-f7ea-40c7-a9b2-b557c213169b admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:02.674797 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094d0dcb-f7ea-40c7-a9b2-b557c213169b admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:02.679306 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc222b82-d5d0-418a-a744-a2712bd492ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:02.680210 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc222b82-d5d0-418a-a744-a2712bd492ee admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:02.680489 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc222b82-d5d0-418a-a744-a2712bd492ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:02.680754 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc222b82-d5d0-418a-a744-a2712bd492ee admin admin] PATH_INFO: `/v3/projects/c28aa28baebe48098c3c3ab89988cb31/users/4183237aa850499987a8b4b01f759253/roles/60af5088793e4379b88821a86552a344` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:02.695300 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc222b82-d5d0-418a-a744-a2712bd492ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=c28aa28baebe48098c3c3ab89988cb31, user_id=4183237aa850499987a8b4b01f759253, role_id=60af5088793e4379b88821a86552a344)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:02.698019 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc222b82-d5d0-418a-a744-a2712bd492ee admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:02.767391 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1266/2534] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:36:02 2022] PUT /identity/v3/projects/c28aa28baebe48098c3c3ab89988cb31/users/4183237aa850499987a8b4b01f759253/roles/60af5088793e4379b88821a86552a344 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 17 19:36:02.771428 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60a2f53-8ac5-47d1-907d-dd6722800905 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:02.781225 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60a2f53-8ac5-47d1-907d-dd6722800905 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:02.835539 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8763b28-f4fa-4927-9521-a81f11127d64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:02.835911 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8763b28-f4fa-4927-9521-a81f11127d64 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:02.836188 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8763b28-f4fa-4927-9521-a81f11127d64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:02.836457 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8763b28-f4fa-4927-9521-a81f11127d64 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:02.837173 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8763b28-f4fa-4927-9521-a81f11127d64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:02.838746 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8763b28-f4fa-4927-9521-a81f11127d64 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:02.848757 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1269/2535] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:36:02 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:36:02.852890 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc222b82-d5d0-418a-a744-a2712bd492ee admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:02.864567 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc222b82-d5d0-418a-a744-a2712bd492ee admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:02.870782 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f20ba26-3674-460c-bed6-34820227a073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:02.871744 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f20ba26-3674-460c-bed6-34820227a073 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:02.872135 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f20ba26-3674-460c-bed6-34820227a073 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:02.872413 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f20ba26-3674-460c-bed6-34820227a073 admin admin] PATH_INFO: `/v3/projects/c28aa28baebe48098c3c3ab89988cb31/users/4183237aa850499987a8b4b01f759253/roles/60af5088793e4379b88821a86552a344` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:02.874361 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f20ba26-3674-460c-bed6-34820227a073 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c28aa28baebe48098c3c3ab89988cb31, user_id=4183237aa850499987a8b4b01f759253, role_id=60af5088793e4379b88821a86552a344)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:02.877037 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f20ba26-3674-460c-bed6-34820227a073 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:02.887677 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1267/2536] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:36:02 2022] PUT /identity/v3/projects/c28aa28baebe48098c3c3ab89988cb31/users/4183237aa850499987a8b4b01f759253/roles/60af5088793e4379b88821a86552a344 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:36:02.898108 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7254ea64-9466-49b0-b331-2230ae90e4b0 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:02.898395 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7254ea64-9466-49b0-b331-2230ae90e4b0 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:02.898669 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7254ea64-9466-49b0-b331-2230ae90e4b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:02.924754 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-7254ea64-9466-49b0-b331-2230ae90e4b0 None None] MFA Rules not processed for user `4183237aa850499987a8b4b01f759253`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:36:02.926177 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-7254ea64-9466-49b0-b331-2230ae90e4b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:36:02.994323 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1270/2537] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:36:02 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:36:03.006740 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f20ba26-3674-460c-bed6-34820227a073 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:03.008495 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-6f20ba26-3674-460c-bed6-34820227a073 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:36:03.067068 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f20ba26-3674-460c-bed6-34820227a073 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:03.070715 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4e974d-ad27-4cf3-9d5f-07af7233ba23 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd05409b38ae346458abfe41ebda1c59e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:03.071370 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4e974d-ad27-4cf3-9d5f-07af7233ba23 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:03.071683 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4e974d-ad27-4cf3-9d5f-07af7233ba23 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:03.071956 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4e974d-ad27-4cf3-9d5f-07af7233ba23 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:03.081716 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4e974d-ad27-4cf3-9d5f-07af7233ba23 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:03.083182 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4e974d-ad27-4cf3-9d5f-07af7233ba23 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:03.093772 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1268/2538] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:36:03 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:36:03.294738 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7254ea64-9466-49b0-b331-2230ae90e4b0 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:03.304320 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7254ea64-9466-49b0-b331-2230ae90e4b0 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:03.308125 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ffbbc2-4d7c-43b3-8922-5a461d6965c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd05409b38ae346458abfe41ebda1c59e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:03.308769 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ffbbc2-4d7c-43b3-8922-5a461d6965c7 service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:03.309052 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ffbbc2-4d7c-43b3-8922-5a461d6965c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:03.309330 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ffbbc2-4d7c-43b3-8922-5a461d6965c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:03.364770 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ffbbc2-4d7c-43b3-8922-5a461d6965c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:03.366117 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ffbbc2-4d7c-43b3-8922-5a461d6965c7 service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:03.374829 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1271/2539] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:36:03 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:36:06.844536 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4e974d-ad27-4cf3-9d5f-07af7233ba23 service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:06.904391 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4e974d-ad27-4cf3-9d5f-07af7233ba23 service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:06.908875 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b3a959-3308-48e6-a31a-7fd15fabe008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:06.909723 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b3a959-3308-48e6-a31a-7fd15fabe008 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:06.909999 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b3a959-3308-48e6-a31a-7fd15fabe008 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:06.910267 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b3a959-3308-48e6-a31a-7fd15fabe008 admin admin] PATH_INFO: `/v3/users/3a6c13fea20e4cd98e77043a12ecdecf` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:06.911424 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b3a959-3308-48e6-a31a-7fd15fabe008 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a6c13fea20e4cd98e77043a12ecdecf)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:06.913367 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b3a959-3308-48e6-a31a-7fd15fabe008 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:07.112406 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-83b3a959-3308-48e6-a31a-7fd15fabe008 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a6c13fea20e4cd98e77043a12ecdecf'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:07.112975 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-83b3a959-3308-48e6-a31a-7fd15fabe008 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a6c13fea20e4cd98e77043a12ecdecf'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:07.125507 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-83b3a959-3308-48e6-a31a-7fd15fabe008 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a6c13fea20e4cd98e77043a12ecdecf'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:07.184135 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-83b3a959-3308-48e6-a31a-7fd15fabe008 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a6c13fea20e4cd98e77043a12ecdecf'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:07.191772 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-83b3a959-3308-48e6-a31a-7fd15fabe008 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a6c13fea20e4cd98e77043a12ecdecf'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:07.198825 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1269/2540] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 17 19:36:06 2022] DELETE /identity/v3/users/3a6c13fea20e4cd98e77043a12ecdecf => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:36:07.203352 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929c5af9-11b7-4a47-9801-f343de30a7bb None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:07.203898 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929c5af9-11b7-4a47-9801-f343de30a7bb None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:07.203994 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929c5af9-11b7-4a47-9801-f343de30a7bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:07.220658 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-929c5af9-11b7-4a47-9801-f343de30a7bb None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:36:07.221848 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-929c5af9-11b7-4a47-9801-f343de30a7bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:36:07.280968 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1272/2541] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:36:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:36:07.290501 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b3a959-3308-48e6-a31a-7fd15fabe008 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:07.292448 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-83b3a959-3308-48e6-a31a-7fd15fabe008 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:36:07.346704 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b3a959-3308-48e6-a31a-7fd15fabe008 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:07.350312 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce50edad-74ec-4931-bbe7-dbd3ee7946cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:07.350941 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce50edad-74ec-4931-bbe7-dbd3ee7946cd service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:07.351219 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce50edad-74ec-4931-bbe7-dbd3ee7946cd service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:07.351509 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce50edad-74ec-4931-bbe7-dbd3ee7946cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:07.361182 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce50edad-74ec-4931-bbe7-dbd3ee7946cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:07.362655 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce50edad-74ec-4931-bbe7-dbd3ee7946cd service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:07.372359 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1270/2542] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:36:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:36:07.762174 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929c5af9-11b7-4a47-9801-f343de30a7bb None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:07.763738 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-929c5af9-11b7-4a47-9801-f343de30a7bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:36:07.777046 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929c5af9-11b7-4a47-9801-f343de30a7bb None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:07.786994 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7ab615-b41f-4e34-a2aa-8434440875fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:07.788109 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7ab615-b41f-4e34-a2aa-8434440875fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:07.788396 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7ab615-b41f-4e34-a2aa-8434440875fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:07.788673 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7ab615-b41f-4e34-a2aa-8434440875fc admin admin] PATH_INFO: `/v3/projects/e318052251794abe9cbf360278e40967` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:07.789970 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7ab615-b41f-4e34-a2aa-8434440875fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=e318052251794abe9cbf360278e40967)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:07.792569 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7ab615-b41f-4e34-a2aa-8434440875fc admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:07.884670 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-6f7ab615-b41f-4e34-a2aa-8434440875fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e318052251794abe9cbf360278e40967', 'request_id': 'req-6f7ab615-b41f-4e34-a2aa-8434440875fc'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:07.890803 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-6f7ab615-b41f-4e34-a2aa-8434440875fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e318052251794abe9cbf360278e40967', 'request_id': 'req-6f7ab615-b41f-4e34-a2aa-8434440875fc'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:07.896520 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-6f7ab615-b41f-4e34-a2aa-8434440875fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e318052251794abe9cbf360278e40967', 'request_id': 'req-6f7ab615-b41f-4e34-a2aa-8434440875fc'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:07.966496 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-6f7ab615-b41f-4e34-a2aa-8434440875fc admin admin] The token cache is being invalidate because project e318052251794abe9cbf360278e40967 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 17 19:36:07.967827 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-6f7ab615-b41f-4e34-a2aa-8434440875fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:07.967827 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-6f7ab615-b41f-4e34-a2aa-8434440875fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:07.969120 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1273/2543] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 17 19:36:07 2022] DELETE /identity/v3/projects/e318052251794abe9cbf360278e40967 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:36:07.973078 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce50edad-74ec-4931-bbe7-dbd3ee7946cd service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:07.983770 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce50edad-74ec-4931-bbe7-dbd3ee7946cd service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:08.038034 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8291ff-4a17-45e9-82c6-789375d0bd0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:08.038894 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8291ff-4a17-45e9-82c6-789375d0bd0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:08.039173 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8291ff-4a17-45e9-82c6-789375d0bd0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:08.039446 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8291ff-4a17-45e9-82c6-789375d0bd0b admin admin] PATH_INFO: `/v3/users/4183237aa850499987a8b4b01f759253` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:08.040645 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8291ff-4a17-45e9-82c6-789375d0bd0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=4183237aa850499987a8b4b01f759253)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:08.043191 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8291ff-4a17-45e9-82c6-789375d0bd0b admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:08.248161 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-cb8291ff-4a17-45e9-82c6-789375d0bd0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4183237aa850499987a8b4b01f759253'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:08.248836 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-cb8291ff-4a17-45e9-82c6-789375d0bd0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4183237aa850499987a8b4b01f759253'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:08.260065 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-cb8291ff-4a17-45e9-82c6-789375d0bd0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4183237aa850499987a8b4b01f759253'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:08.326089 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-cb8291ff-4a17-45e9-82c6-789375d0bd0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4183237aa850499987a8b4b01f759253'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:08.332822 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-cb8291ff-4a17-45e9-82c6-789375d0bd0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4183237aa850499987a8b4b01f759253'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:08.339277 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1271/2544] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 17 19:36:07 2022] DELETE /identity/v3/users/4183237aa850499987a8b4b01f759253 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:36:08.739465 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7ab615-b41f-4e34-a2aa-8434440875fc admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:08.741039 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-6f7ab615-b41f-4e34-a2aa-8434440875fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:36:08.756519 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7ab615-b41f-4e34-a2aa-8434440875fc admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:08.817572 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8e77d6-357c-4b82-bb4d-55546724273d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:08.818566 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8e77d6-357c-4b82-bb4d-55546724273d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:08.818904 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8e77d6-357c-4b82-bb4d-55546724273d admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:08.819180 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8e77d6-357c-4b82-bb4d-55546724273d admin admin] PATH_INFO: `/v3/projects/c28aa28baebe48098c3c3ab89988cb31` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:08.820393 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8e77d6-357c-4b82-bb4d-55546724273d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c28aa28baebe48098c3c3ab89988cb31)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:08.822894 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8e77d6-357c-4b82-bb4d-55546724273d admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:08.937939 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-dd8e77d6-357c-4b82-bb4d-55546724273d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c28aa28baebe48098c3c3ab89988cb31', 'request_id': 'req-dd8e77d6-357c-4b82-bb4d-55546724273d'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:08.944209 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-dd8e77d6-357c-4b82-bb4d-55546724273d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c28aa28baebe48098c3c3ab89988cb31', 'request_id': 'req-dd8e77d6-357c-4b82-bb4d-55546724273d'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:08.952309 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-dd8e77d6-357c-4b82-bb4d-55546724273d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c28aa28baebe48098c3c3ab89988cb31', 'request_id': 'req-dd8e77d6-357c-4b82-bb4d-55546724273d'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:09.034393 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-dd8e77d6-357c-4b82-bb4d-55546724273d admin admin] The token cache is being invalidate because project c28aa28baebe48098c3c3ab89988cb31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 17 19:36:09.034838 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-dd8e77d6-357c-4b82-bb4d-55546724273d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:09.035558 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-dd8e77d6-357c-4b82-bb4d-55546724273d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:09.037193 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1274/2545] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 17 19:36:08 2022] DELETE /identity/v3/projects/c28aa28baebe48098c3c3ab89988cb31 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:36:09.042522 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0496f3c-7b2c-4d66-8764-e057a777499f None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:09.043009 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0496f3c-7b2c-4d66-8764-e057a777499f None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:09.043204 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0496f3c-7b2c-4d66-8764-e057a777499f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:09.061394 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-f0496f3c-7b2c-4d66-8764-e057a777499f None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:36:09.062582 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-f0496f3c-7b2c-4d66-8764-e057a777499f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:36:09.121392 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1272/2546] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:36:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:36:09.128888 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a48e77a-632e-4506-8def-8d79dfce646d None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:09.129241 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a48e77a-632e-4506-8def-8d79dfce646d None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:09.129613 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a48e77a-632e-4506-8def-8d79dfce646d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:09.148611 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-8a48e77a-632e-4506-8def-8d79dfce646d None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:36:09.149975 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-8a48e77a-632e-4506-8def-8d79dfce646d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:36:09.158958 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1275/2547] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:36:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:36:09.163102 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0496f3c-7b2c-4d66-8764-e057a777499f None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:09.173552 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0496f3c-7b2c-4d66-8764-e057a777499f None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:09.177869 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfe24f3-9b3a-4d18-9501-d8748d41ee07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:09.178524 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfe24f3-9b3a-4d18-9501-d8748d41ee07 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:09.178801 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfe24f3-9b3a-4d18-9501-d8748d41ee07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:09.179075 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfe24f3-9b3a-4d18-9501-d8748d41ee07 admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:09.179863 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfe24f3-9b3a-4d18-9501-d8748d41ee07 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:36:09.181885 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfe24f3-9b3a-4d18-9501-d8748d41ee07 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:09.181885 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfe24f3-9b3a-4d18-9501-d8748d41ee07 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:09.193693 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1273/2548] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 17 19:36:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:36:09.197970 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a48e77a-632e-4506-8def-8d79dfce646d None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:09.207640 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a48e77a-632e-4506-8def-8d79dfce646d None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:09.212251 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a733d9-41a4-46c5-be82-cc5c5ce9556a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:09.212884 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a733d9-41a4-46c5-be82-cc5c5ce9556a admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:09.213160 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a733d9-41a4-46c5-be82-cc5c5ce9556a admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:09.213427 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a733d9-41a4-46c5-be82-cc5c5ce9556a admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:09.214667 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a733d9-41a4-46c5-be82-cc5c5ce9556a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:09.216681 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a733d9-41a4-46c5-be82-cc5c5ce9556a admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:09.354838 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1276/2549] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 17 19:36:09 2022] POST /identity/v3/projects => generated 401 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:36:09.358745 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfe24f3-9b3a-4d18-9501-d8748d41ee07 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:09.368569 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfe24f3-9b3a-4d18-9501-d8748d41ee07 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:09.423458 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dea820-7189-4fba-b2af-b5ba436e8f0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:09.424147 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dea820-7189-4fba-b2af-b5ba436e8f0a admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:09.424426 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dea820-7189-4fba-b2af-b5ba436e8f0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:09.424698 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dea820-7189-4fba-b2af-b5ba436e8f0a admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:09.425381 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dea820-7189-4fba-b2af-b5ba436e8f0a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:09.427333 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dea820-7189-4fba-b2af-b5ba436e8f0a admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:09.526767 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1274/2550] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 17 19:36:09 2022] POST /identity/v3/users => generated 301 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:36:09.530921 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a733d9-41a4-46c5-be82-cc5c5ce9556a admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:09.541190 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a733d9-41a4-46c5-be82-cc5c5ce9556a admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:09.546752 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d1d8d2-e296-4908-9167-e1f355b63f56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:09.547396 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d1d8d2-e296-4908-9167-e1f355b63f56 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:09.547796 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d1d8d2-e296-4908-9167-e1f355b63f56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:09.548122 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d1d8d2-e296-4908-9167-e1f355b63f56 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:09.548867 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d1d8d2-e296-4908-9167-e1f355b63f56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:09.550450 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d1d8d2-e296-4908-9167-e1f355b63f56 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:09.560676 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1277/2551] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:36:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:36:09.564626 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dea820-7189-4fba-b2af-b5ba436e8f0a admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:09.574964 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dea820-7189-4fba-b2af-b5ba436e8f0a admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:09.579597 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909c71fe-04f8-4f8b-b6f0-72cf19b5df98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:09.580241 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909c71fe-04f8-4f8b-b6f0-72cf19b5df98 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:09.580520 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909c71fe-04f8-4f8b-b6f0-72cf19b5df98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:09.580879 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909c71fe-04f8-4f8b-b6f0-72cf19b5df98 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:09.581616 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909c71fe-04f8-4f8b-b6f0-72cf19b5df98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:09.583248 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909c71fe-04f8-4f8b-b6f0-72cf19b5df98 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:09.593712 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1275/2552] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:36:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:36:09.597652 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d1d8d2-e296-4908-9167-e1f355b63f56 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:09.607780 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d1d8d2-e296-4908-9167-e1f355b63f56 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:09.612317 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4583c6c5-fc8b-4382-b09f-e1bbe839df15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:09.613172 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4583c6c5-fc8b-4382-b09f-e1bbe839df15 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:09.616143 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4583c6c5-fc8b-4382-b09f-e1bbe839df15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:09.616143 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4583c6c5-fc8b-4382-b09f-e1bbe839df15 admin admin] PATH_INFO: `/v3/projects/a74363fc90aa441f9874b87453479134/users/7ace567df5f8487e86f6111aa7d500a9/roles/114e6a16e63447d1924707ace5aace25` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:09.627660 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4583c6c5-fc8b-4382-b09f-e1bbe839df15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a74363fc90aa441f9874b87453479134, user_id=7ace567df5f8487e86f6111aa7d500a9, role_id=114e6a16e63447d1924707ace5aace25)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:09.630341 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4583c6c5-fc8b-4382-b09f-e1bbe839df15 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:09.697650 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1278/2553] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:36:09 2022] PUT /identity/v3/projects/a74363fc90aa441f9874b87453479134/users/7ace567df5f8487e86f6111aa7d500a9/roles/114e6a16e63447d1924707ace5aace25 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:36:09.709142 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba3828e-222d-487d-921c-e72ebb6c260a None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:09.716272 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba3828e-222d-487d-921c-e72ebb6c260a None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:09.716272 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba3828e-222d-487d-921c-e72ebb6c260a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:09.736103 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-dba3828e-222d-487d-921c-e72ebb6c260a None None] MFA Rules not processed for user `7ace567df5f8487e86f6111aa7d500a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:36:09.737308 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-dba3828e-222d-487d-921c-e72ebb6c260a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:36:09.799728 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1276/2554] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:36:09 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:36:09.804130 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4583c6c5-fc8b-4382-b09f-e1bbe839df15 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:09.862623 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4583c6c5-fc8b-4382-b09f-e1bbe839df15 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:09.867047 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4c82e8-8122-4233-beef-71e366244db8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:09.867739 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4c82e8-8122-4233-beef-71e366244db8 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:09.868018 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4c82e8-8122-4233-beef-71e366244db8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:09.868284 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4c82e8-8122-4233-beef-71e366244db8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:09.868963 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4c82e8-8122-4233-beef-71e366244db8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:09.870895 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4c82e8-8122-4233-beef-71e366244db8 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:09.966147 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1279/2555] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 17 19:36:09 2022] POST /identity/v3/projects => generated 401 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 17 19:36:09.969957 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba3828e-222d-487d-921c-e72ebb6c260a None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:09.981458 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba3828e-222d-487d-921c-e72ebb6c260a None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:10.035303 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb5a16f-ee77-4c2f-bc4b-21e81e329115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:10.035987 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb5a16f-ee77-4c2f-bc4b-21e81e329115 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:10.036264 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb5a16f-ee77-4c2f-bc4b-21e81e329115 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:10.036534 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb5a16f-ee77-4c2f-bc4b-21e81e329115 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:10.037207 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb5a16f-ee77-4c2f-bc4b-21e81e329115 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:10.039146 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb5a16f-ee77-4c2f-bc4b-21e81e329115 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:10.137088 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1277/2556] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 17 19:36:09 2022] POST /identity/v3/users => generated 307 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:36:10.140980 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4c82e8-8122-4233-beef-71e366244db8 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:10.151365 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4c82e8-8122-4233-beef-71e366244db8 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:10.156423 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a87cf13-066c-4995-8917-74708b90c3f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:10.157077 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a87cf13-066c-4995-8917-74708b90c3f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:10.157386 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a87cf13-066c-4995-8917-74708b90c3f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:10.157662 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a87cf13-066c-4995-8917-74708b90c3f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:10.158389 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a87cf13-066c-4995-8917-74708b90c3f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:10.160039 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a87cf13-066c-4995-8917-74708b90c3f8 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:10.170822 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1280/2557] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:36:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:36:10.174782 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb5a16f-ee77-4c2f-bc4b-21e81e329115 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:10.185464 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb5a16f-ee77-4c2f-bc4b-21e81e329115 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:10.192121 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56de4bb-de28-4883-afd4-0f665e00c0bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:10.194399 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56de4bb-de28-4883-afd4-0f665e00c0bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:10.194772 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56de4bb-de28-4883-afd4-0f665e00c0bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:10.195140 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56de4bb-de28-4883-afd4-0f665e00c0bf admin admin] PATH_INFO: `/v3/projects/68a81ebc9008421f89a9af98ae3e87c4/users/8920f02999d644de92eaae34b3c5d235/roles/60af5088793e4379b88821a86552a344` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:10.210784 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56de4bb-de28-4883-afd4-0f665e00c0bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=68a81ebc9008421f89a9af98ae3e87c4, user_id=8920f02999d644de92eaae34b3c5d235, role_id=60af5088793e4379b88821a86552a344)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:10.213676 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56de4bb-de28-4883-afd4-0f665e00c0bf admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:10.284889 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1278/2558] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:36:10 2022] PUT /identity/v3/projects/68a81ebc9008421f89a9af98ae3e87c4/users/8920f02999d644de92eaae34b3c5d235/roles/60af5088793e4379b88821a86552a344 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:36:10.288701 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a87cf13-066c-4995-8917-74708b90c3f8 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:10.298807 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a87cf13-066c-4995-8917-74708b90c3f8 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:10.351782 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb00cfa0-9c4b-4784-8f28-b667a89e927d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:10.352431 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb00cfa0-9c4b-4784-8f28-b667a89e927d admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:10.352710 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb00cfa0-9c4b-4784-8f28-b667a89e927d admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:10.352985 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb00cfa0-9c4b-4784-8f28-b667a89e927d admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:10.353960 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb00cfa0-9c4b-4784-8f28-b667a89e927d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:10.356060 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb00cfa0-9c4b-4784-8f28-b667a89e927d admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:10.365410 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1281/2559] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:36:10 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:36:10.369540 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56de4bb-de28-4883-afd4-0f665e00c0bf admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:10.380411 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56de4bb-de28-4883-afd4-0f665e00c0bf admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:10.385076 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d144d3-a1d1-4944-b0e6-7a49660a88d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:10.385939 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d144d3-a1d1-4944-b0e6-7a49660a88d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:10.386216 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d144d3-a1d1-4944-b0e6-7a49660a88d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:10.386490 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d144d3-a1d1-4944-b0e6-7a49660a88d4 admin admin] PATH_INFO: `/v3/projects/68a81ebc9008421f89a9af98ae3e87c4/users/8920f02999d644de92eaae34b3c5d235/roles/60af5088793e4379b88821a86552a344` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:10.388543 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d144d3-a1d1-4944-b0e6-7a49660a88d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68a81ebc9008421f89a9af98ae3e87c4, user_id=8920f02999d644de92eaae34b3c5d235, role_id=60af5088793e4379b88821a86552a344)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:10.391131 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d144d3-a1d1-4944-b0e6-7a49660a88d4 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:10.400995 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1279/2560] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:36:10 2022] PUT /identity/v3/projects/68a81ebc9008421f89a9af98ae3e87c4/users/8920f02999d644de92eaae34b3c5d235/roles/60af5088793e4379b88821a86552a344 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 17 19:36:10.411005 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46773df4-9185-46fd-89d7-a20b49aa539e None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:10.411298 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46773df4-9185-46fd-89d7-a20b49aa539e None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:10.411607 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46773df4-9185-46fd-89d7-a20b49aa539e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:10.437719 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-46773df4-9185-46fd-89d7-a20b49aa539e None None] MFA Rules not processed for user `8920f02999d644de92eaae34b3c5d235`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:36:10.438902 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-46773df4-9185-46fd-89d7-a20b49aa539e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:36:10.502917 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1282/2561] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:36:10 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:36:10.514340 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d144d3-a1d1-4944-b0e6-7a49660a88d4 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:10.515994 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-00d144d3-a1d1-4944-b0e6-7a49660a88d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:36:10.571006 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d144d3-a1d1-4944-b0e6-7a49660a88d4 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:10.575179 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4674a5b-96e8-4dbb-8843-fac01ca7adcf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd05409b38ae346458abfe41ebda1c59e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:10.575850 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4674a5b-96e8-4dbb-8843-fac01ca7adcf service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:10.576130 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4674a5b-96e8-4dbb-8843-fac01ca7adcf service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:10.576397 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4674a5b-96e8-4dbb-8843-fac01ca7adcf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:10.585333 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4674a5b-96e8-4dbb-8843-fac01ca7adcf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:10.586756 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4674a5b-96e8-4dbb-8843-fac01ca7adcf service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:10.597034 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1280/2562] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:36:10 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:36:10.809121 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46773df4-9185-46fd-89d7-a20b49aa539e None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:10.818656 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46773df4-9185-46fd-89d7-a20b49aa539e None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:10.822198 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26993b0a-6f0c-42af-9133-95a9a9e9afc2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd05409b38ae346458abfe41ebda1c59e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:10.822822 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26993b0a-6f0c-42af-9133-95a9a9e9afc2 service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:10.823740 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26993b0a-6f0c-42af-9133-95a9a9e9afc2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:10.824013 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26993b0a-6f0c-42af-9133-95a9a9e9afc2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:10.879616 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26993b0a-6f0c-42af-9133-95a9a9e9afc2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:10.881050 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26993b0a-6f0c-42af-9133-95a9a9e9afc2 service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:10.890634 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1283/2563] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:36:10 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:36:15.910967 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4674a5b-96e8-4dbb-8843-fac01ca7adcf service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:15.971958 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4674a5b-96e8-4dbb-8843-fac01ca7adcf service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:15.976596 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617953e5-17d5-4568-abfb-af253981d33f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:15.977459 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617953e5-17d5-4568-abfb-af253981d33f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:15.979306 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617953e5-17d5-4568-abfb-af253981d33f admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:15.979306 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617953e5-17d5-4568-abfb-af253981d33f admin admin] PATH_INFO: `/v3/users/7ace567df5f8487e86f6111aa7d500a9` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:15.979794 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617953e5-17d5-4568-abfb-af253981d33f admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ace567df5f8487e86f6111aa7d500a9)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:15.981429 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617953e5-17d5-4568-abfb-af253981d33f admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:16.150658 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-617953e5-17d5-4568-abfb-af253981d33f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ace567df5f8487e86f6111aa7d500a9'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:16.151298 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-617953e5-17d5-4568-abfb-af253981d33f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ace567df5f8487e86f6111aa7d500a9'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:16.162774 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-617953e5-17d5-4568-abfb-af253981d33f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ace567df5f8487e86f6111aa7d500a9'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:16.234380 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-617953e5-17d5-4568-abfb-af253981d33f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ace567df5f8487e86f6111aa7d500a9'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:16.241634 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-617953e5-17d5-4568-abfb-af253981d33f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ace567df5f8487e86f6111aa7d500a9'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:16.248217 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1281/2564] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 17 19:36:15 2022] DELETE /identity/v3/users/7ace567df5f8487e86f6111aa7d500a9 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:36:16.252675 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f9b468-d4bc-4583-a293-80dcb35e253f None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:16.253135 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f9b468-d4bc-4583-a293-80dcb35e253f None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:16.253229 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f9b468-d4bc-4583-a293-80dcb35e253f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:16.270404 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-50f9b468-d4bc-4583-a293-80dcb35e253f None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:36:16.271627 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-50f9b468-d4bc-4583-a293-80dcb35e253f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:36:16.328020 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1284/2565] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:36:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:36:16.337445 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617953e5-17d5-4568-abfb-af253981d33f admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:16.338899 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-617953e5-17d5-4568-abfb-af253981d33f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:36:16.394176 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617953e5-17d5-4568-abfb-af253981d33f admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:16.397793 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ed1c9b-fbfc-4e39-8657-6ea568b92bd8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:16.398415 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ed1c9b-fbfc-4e39-8657-6ea568b92bd8 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:16.398690 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ed1c9b-fbfc-4e39-8657-6ea568b92bd8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:16.398959 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ed1c9b-fbfc-4e39-8657-6ea568b92bd8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:16.407972 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ed1c9b-fbfc-4e39-8657-6ea568b92bd8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:16.409327 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ed1c9b-fbfc-4e39-8657-6ea568b92bd8 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:16.419720 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1282/2566] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:36:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:36:16.783230 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f9b468-d4bc-4583-a293-80dcb35e253f None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:16.784779 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-50f9b468-d4bc-4583-a293-80dcb35e253f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:36:16.798294 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f9b468-d4bc-4583-a293-80dcb35e253f None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:16.802988 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf701b90-aa4e-4451-8b6b-00e23c3fdfea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:16.803868 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf701b90-aa4e-4451-8b6b-00e23c3fdfea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:16.804144 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf701b90-aa4e-4451-8b6b-00e23c3fdfea admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:16.804422 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf701b90-aa4e-4451-8b6b-00e23c3fdfea admin admin] PATH_INFO: `/v3/projects/a74363fc90aa441f9874b87453479134` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:16.805591 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf701b90-aa4e-4451-8b6b-00e23c3fdfea admin admin] RBAC: Authorizing `identity:delete_project(project_id=a74363fc90aa441f9874b87453479134)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:16.807512 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf701b90-aa4e-4451-8b6b-00e23c3fdfea admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:16.907131 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-bf701b90-aa4e-4451-8b6b-00e23c3fdfea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a74363fc90aa441f9874b87453479134', 'request_id': 'req-bf701b90-aa4e-4451-8b6b-00e23c3fdfea'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:16.918114 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-bf701b90-aa4e-4451-8b6b-00e23c3fdfea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a74363fc90aa441f9874b87453479134', 'request_id': 'req-bf701b90-aa4e-4451-8b6b-00e23c3fdfea'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:16.919367 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-bf701b90-aa4e-4451-8b6b-00e23c3fdfea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a74363fc90aa441f9874b87453479134', 'request_id': 'req-bf701b90-aa4e-4451-8b6b-00e23c3fdfea'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:16.983420 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-bf701b90-aa4e-4451-8b6b-00e23c3fdfea admin admin] The token cache is being invalidate because project a74363fc90aa441f9874b87453479134 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 17 19:36:16.983740 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-bf701b90-aa4e-4451-8b6b-00e23c3fdfea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:16.984361 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-bf701b90-aa4e-4451-8b6b-00e23c3fdfea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:16.985905 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1285/2567] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 17 19:36:16 2022] DELETE /identity/v3/projects/a74363fc90aa441f9874b87453479134 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:36:16.990071 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ed1c9b-fbfc-4e39-8657-6ea568b92bd8 service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:16.999907 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ed1c9b-fbfc-4e39-8657-6ea568b92bd8 service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:17.055142 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcff7d3d-2d42-460d-ba52-a8e9538706ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:17.056021 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcff7d3d-2d42-460d-ba52-a8e9538706ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:17.056301 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcff7d3d-2d42-460d-ba52-a8e9538706ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:17.056572 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcff7d3d-2d42-460d-ba52-a8e9538706ea admin admin] PATH_INFO: `/v3/users/8920f02999d644de92eaae34b3c5d235` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:17.057729 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcff7d3d-2d42-460d-ba52-a8e9538706ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=8920f02999d644de92eaae34b3c5d235)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:17.059723 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcff7d3d-2d42-460d-ba52-a8e9538706ea admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:17.214224 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-fcff7d3d-2d42-460d-ba52-a8e9538706ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8920f02999d644de92eaae34b3c5d235'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:17.215468 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-fcff7d3d-2d42-460d-ba52-a8e9538706ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8920f02999d644de92eaae34b3c5d235'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:17.226804 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-fcff7d3d-2d42-460d-ba52-a8e9538706ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8920f02999d644de92eaae34b3c5d235'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:17.303715 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-fcff7d3d-2d42-460d-ba52-a8e9538706ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8920f02999d644de92eaae34b3c5d235'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:17.310882 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-fcff7d3d-2d42-460d-ba52-a8e9538706ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8920f02999d644de92eaae34b3c5d235'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:17.316708 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1283/2568] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 17 19:36:16 2022] DELETE /identity/v3/users/8920f02999d644de92eaae34b3c5d235 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:36:17.731585 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf701b90-aa4e-4451-8b6b-00e23c3fdfea admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:17.733009 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-bf701b90-aa4e-4451-8b6b-00e23c3fdfea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:36:17.746764 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf701b90-aa4e-4451-8b6b-00e23c3fdfea admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:17.803394 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a438dc-b110-41f1-a8bc-6bf452d04610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:17.804274 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a438dc-b110-41f1-a8bc-6bf452d04610 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:17.804707 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a438dc-b110-41f1-a8bc-6bf452d04610 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:17.804967 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a438dc-b110-41f1-a8bc-6bf452d04610 admin admin] PATH_INFO: `/v3/projects/68a81ebc9008421f89a9af98ae3e87c4` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:17.806126 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a438dc-b110-41f1-a8bc-6bf452d04610 admin admin] RBAC: Authorizing `identity:delete_project(project_id=68a81ebc9008421f89a9af98ae3e87c4)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:17.808674 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a438dc-b110-41f1-a8bc-6bf452d04610 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:17.906300 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-f5a438dc-b110-41f1-a8bc-6bf452d04610 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68a81ebc9008421f89a9af98ae3e87c4', 'request_id': 'req-f5a438dc-b110-41f1-a8bc-6bf452d04610'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:17.912243 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-f5a438dc-b110-41f1-a8bc-6bf452d04610 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68a81ebc9008421f89a9af98ae3e87c4', 'request_id': 'req-f5a438dc-b110-41f1-a8bc-6bf452d04610'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:17.917991 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-f5a438dc-b110-41f1-a8bc-6bf452d04610 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68a81ebc9008421f89a9af98ae3e87c4', 'request_id': 'req-f5a438dc-b110-41f1-a8bc-6bf452d04610'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:18.003802 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-f5a438dc-b110-41f1-a8bc-6bf452d04610 admin admin] The token cache is being invalidate because project 68a81ebc9008421f89a9af98ae3e87c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 17 19:36:18.004094 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-f5a438dc-b110-41f1-a8bc-6bf452d04610 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:18.004703 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-f5a438dc-b110-41f1-a8bc-6bf452d04610 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:18.006274 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1286/2569] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 17 19:36:17 2022] DELETE /identity/v3/projects/68a81ebc9008421f89a9af98ae3e87c4 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:36:18.011434 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb9745c-acab-4365-898f-400b617810aa None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:18.011907 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb9745c-acab-4365-898f-400b617810aa None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:18.012016 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb9745c-acab-4365-898f-400b617810aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:18.029032 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-0bb9745c-acab-4365-898f-400b617810aa None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:36:18.030208 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-0bb9745c-acab-4365-898f-400b617810aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:36:18.085996 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1284/2570] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:36:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:36:18.094536 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4528511d-a5da-431a-a8ce-53d3408d75f4 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:18.094819 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4528511d-a5da-431a-a8ce-53d3408d75f4 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:18.095087 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4528511d-a5da-431a-a8ce-53d3408d75f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:18.113601 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-4528511d-a5da-431a-a8ce-53d3408d75f4 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:36:18.114825 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-4528511d-a5da-431a-a8ce-53d3408d75f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:36:18.123231 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1287/2571] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:36:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:36:18.127199 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb9745c-acab-4365-898f-400b617810aa None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:18.136455 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb9745c-acab-4365-898f-400b617810aa None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:18.140909 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cb2069-c137-4f25-89d3-28da1be1d8ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:18.141538 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cb2069-c137-4f25-89d3-28da1be1d8ca admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:18.141814 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cb2069-c137-4f25-89d3-28da1be1d8ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:18.142082 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cb2069-c137-4f25-89d3-28da1be1d8ca admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:18.142785 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cb2069-c137-4f25-89d3-28da1be1d8ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:36:18.143079 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cb2069-c137-4f25-89d3-28da1be1d8ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:18.144714 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cb2069-c137-4f25-89d3-28da1be1d8ca admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:18.156758 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1285/2572] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 17 19:36:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:36:18.160565 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4528511d-a5da-431a-a8ce-53d3408d75f4 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:18.169934 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4528511d-a5da-431a-a8ce-53d3408d75f4 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:18.174528 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d88208-7e72-47cf-af76-84ac29f42ddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:18.175155 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d88208-7e72-47cf-af76-84ac29f42ddc admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:18.175432 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d88208-7e72-47cf-af76-84ac29f42ddc admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:18.175738 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d88208-7e72-47cf-af76-84ac29f42ddc admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:18.176424 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d88208-7e72-47cf-af76-84ac29f42ddc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:18.178289 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d88208-7e72-47cf-af76-84ac29f42ddc admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:18.264502 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1288/2573] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 17 19:36:18 2022] POST /identity/v3/projects => generated 389 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 17 19:36:18.268436 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cb2069-c137-4f25-89d3-28da1be1d8ca admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:18.278883 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cb2069-c137-4f25-89d3-28da1be1d8ca admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:18.333601 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff78204-d608-45e8-9405-e0d0e39bff32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:18.334262 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff78204-d608-45e8-9405-e0d0e39bff32 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:18.334537 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff78204-d608-45e8-9405-e0d0e39bff32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:18.334807 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff78204-d608-45e8-9405-e0d0e39bff32 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:18.335470 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff78204-d608-45e8-9405-e0d0e39bff32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:18.337412 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff78204-d608-45e8-9405-e0d0e39bff32 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:18.456291 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1286/2574] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 17 19:36:18 2022] POST /identity/v3/users => generated 295 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:36:18.460352 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d88208-7e72-47cf-af76-84ac29f42ddc admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:18.471278 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d88208-7e72-47cf-af76-84ac29f42ddc admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:18.475861 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781d6e58-954b-41e6-995d-59d02d5e6417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:18.476502 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781d6e58-954b-41e6-995d-59d02d5e6417 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:18.476781 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781d6e58-954b-41e6-995d-59d02d5e6417 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:18.477050 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781d6e58-954b-41e6-995d-59d02d5e6417 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:18.477761 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781d6e58-954b-41e6-995d-59d02d5e6417 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:18.479659 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781d6e58-954b-41e6-995d-59d02d5e6417 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:18.490578 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1289/2575] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:36:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:36:18.494517 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff78204-d608-45e8-9405-e0d0e39bff32 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:18.504473 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff78204-d608-45e8-9405-e0d0e39bff32 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:18.509180 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95a5bf0-dfb6-49a7-95e7-6f0dc5dd8e49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:18.509840 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95a5bf0-dfb6-49a7-95e7-6f0dc5dd8e49 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:18.510103 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95a5bf0-dfb6-49a7-95e7-6f0dc5dd8e49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:18.510381 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95a5bf0-dfb6-49a7-95e7-6f0dc5dd8e49 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:18.511596 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95a5bf0-dfb6-49a7-95e7-6f0dc5dd8e49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:18.513425 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95a5bf0-dfb6-49a7-95e7-6f0dc5dd8e49 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:18.523318 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1287/2576] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:36:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:36:18.527875 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781d6e58-954b-41e6-995d-59d02d5e6417 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:18.537629 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781d6e58-954b-41e6-995d-59d02d5e6417 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:18.542144 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90cdbbf-3c97-437c-b672-4f5170ee44cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:18.542998 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90cdbbf-3c97-437c-b672-4f5170ee44cb admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:18.543277 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90cdbbf-3c97-437c-b672-4f5170ee44cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:18.543596 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90cdbbf-3c97-437c-b672-4f5170ee44cb admin admin] PATH_INFO: `/v3/projects/b32073e696c740e3995fef607bcc3a92/users/ed7f239fe1e54f2e8d8d29272955079a/roles/114e6a16e63447d1924707ace5aace25` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:18.557560 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90cdbbf-3c97-437c-b672-4f5170ee44cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=b32073e696c740e3995fef607bcc3a92, user_id=ed7f239fe1e54f2e8d8d29272955079a, role_id=114e6a16e63447d1924707ace5aace25)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:18.560393 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90cdbbf-3c97-437c-b672-4f5170ee44cb admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:18.658463 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1290/2577] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:36:18 2022] PUT /identity/v3/projects/b32073e696c740e3995fef607bcc3a92/users/ed7f239fe1e54f2e8d8d29272955079a/roles/114e6a16e63447d1924707ace5aace25 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 17 19:36:18.670670 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b9c0f8-36a5-4433-979e-acf990fee174 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:18.670960 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b9c0f8-36a5-4433-979e-acf990fee174 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:18.671236 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b9c0f8-36a5-4433-979e-acf990fee174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:18.696952 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-23b9c0f8-36a5-4433-979e-acf990fee174 None None] MFA Rules not processed for user `ed7f239fe1e54f2e8d8d29272955079a`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:36:18.698138 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-23b9c0f8-36a5-4433-979e-acf990fee174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:36:18.759130 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1288/2578] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:36:18 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:36:18.764022 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90cdbbf-3c97-437c-b672-4f5170ee44cb admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:18.822792 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90cdbbf-3c97-437c-b672-4f5170ee44cb admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:18.827881 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986947ce-3f41-4a5a-88aa-44a298eb9445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:18.828524 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986947ce-3f41-4a5a-88aa-44a298eb9445 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:18.828805 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986947ce-3f41-4a5a-88aa-44a298eb9445 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:18.829080 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986947ce-3f41-4a5a-88aa-44a298eb9445 admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:18.829762 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986947ce-3f41-4a5a-88aa-44a298eb9445 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:18.831667 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986947ce-3f41-4a5a-88aa-44a298eb9445 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:18.956848 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1291/2579] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 17 19:36:18 2022] POST /identity/v3/projects => generated 387 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:36:18.960936 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b9c0f8-36a5-4433-979e-acf990fee174 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:18.971525 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b9c0f8-36a5-4433-979e-acf990fee174 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:19.025765 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccb902b-b3b8-4c49-8020-c5afb4c66079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:19.026422 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccb902b-b3b8-4c49-8020-c5afb4c66079 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:19.026701 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccb902b-b3b8-4c49-8020-c5afb4c66079 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:19.026999 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccb902b-b3b8-4c49-8020-c5afb4c66079 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:19.027720 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccb902b-b3b8-4c49-8020-c5afb4c66079 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:19.029643 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccb902b-b3b8-4c49-8020-c5afb4c66079 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:19.138088 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1289/2580] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 17 19:36:18 2022] POST /identity/v3/users => generated 300 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:36:19.141952 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986947ce-3f41-4a5a-88aa-44a298eb9445 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:19.152185 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986947ce-3f41-4a5a-88aa-44a298eb9445 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:19.156902 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0666516-5344-4ba8-9f45-3c22831f3864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:19.157546 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0666516-5344-4ba8-9f45-3c22831f3864 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:19.157826 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0666516-5344-4ba8-9f45-3c22831f3864 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:19.158097 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0666516-5344-4ba8-9f45-3c22831f3864 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:19.158806 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0666516-5344-4ba8-9f45-3c22831f3864 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:19.160424 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0666516-5344-4ba8-9f45-3c22831f3864 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:19.170643 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1292/2581] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:36:19 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:36:19.174322 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccb902b-b3b8-4c49-8020-c5afb4c66079 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:19.184528 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccb902b-b3b8-4c49-8020-c5afb4c66079 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:19.189027 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d78d55-3dad-47f8-8d33-f16df5a8edde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:19.189886 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d78d55-3dad-47f8-8d33-f16df5a8edde admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:19.190166 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d78d55-3dad-47f8-8d33-f16df5a8edde admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:19.190437 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d78d55-3dad-47f8-8d33-f16df5a8edde admin admin] PATH_INFO: `/v3/projects/5862c9f88bbb43559031d065f373a1f6/users/683671c03e5b467793e01895bd5501d4/roles/60af5088793e4379b88821a86552a344` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:19.205068 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d78d55-3dad-47f8-8d33-f16df5a8edde admin admin] RBAC: Authorizing `identity:create_grant(project_id=5862c9f88bbb43559031d065f373a1f6, user_id=683671c03e5b467793e01895bd5501d4, role_id=60af5088793e4379b88821a86552a344)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:19.207746 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d78d55-3dad-47f8-8d33-f16df5a8edde admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:19.268485 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1290/2582] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:36:19 2022] PUT /identity/v3/projects/5862c9f88bbb43559031d065f373a1f6/users/683671c03e5b467793e01895bd5501d4/roles/60af5088793e4379b88821a86552a344 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:36:19.272318 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0666516-5344-4ba8-9f45-3c22831f3864 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:19.282074 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0666516-5344-4ba8-9f45-3c22831f3864 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:19.337375 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8042df17-21cb-4441-9a75-2ec99ab88a32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:19.338053 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8042df17-21cb-4441-9a75-2ec99ab88a32 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:19.338338 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8042df17-21cb-4441-9a75-2ec99ab88a32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:19.338614 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8042df17-21cb-4441-9a75-2ec99ab88a32 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:19.339337 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8042df17-21cb-4441-9a75-2ec99ab88a32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:19.341074 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8042df17-21cb-4441-9a75-2ec99ab88a32 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:19.351061 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1293/2583] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:36:19 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:36:19.355981 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d78d55-3dad-47f8-8d33-f16df5a8edde admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:19.366111 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d78d55-3dad-47f8-8d33-f16df5a8edde admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:19.371176 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd91d80-fc59-491b-b1d4-0c9f2e953899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:19.372082 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd91d80-fc59-491b-b1d4-0c9f2e953899 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:19.372364 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd91d80-fc59-491b-b1d4-0c9f2e953899 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:19.372635 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd91d80-fc59-491b-b1d4-0c9f2e953899 admin admin] PATH_INFO: `/v3/projects/5862c9f88bbb43559031d065f373a1f6/users/683671c03e5b467793e01895bd5501d4/roles/60af5088793e4379b88821a86552a344` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:19.374592 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd91d80-fc59-491b-b1d4-0c9f2e953899 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5862c9f88bbb43559031d065f373a1f6, user_id=683671c03e5b467793e01895bd5501d4, role_id=60af5088793e4379b88821a86552a344)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:19.377298 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd91d80-fc59-491b-b1d4-0c9f2e953899 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:19.387383 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1291/2584] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:36:19 2022] PUT /identity/v3/projects/5862c9f88bbb43559031d065f373a1f6/users/683671c03e5b467793e01895bd5501d4/roles/60af5088793e4379b88821a86552a344 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:36:19.397511 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b5c669-f3f1-4060-9355-2ef07bb661cd None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:19.397736 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b5c669-f3f1-4060-9355-2ef07bb661cd None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:19.398010 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b5c669-f3f1-4060-9355-2ef07bb661cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:19.423399 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-29b5c669-f3f1-4060-9355-2ef07bb661cd None None] MFA Rules not processed for user `683671c03e5b467793e01895bd5501d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:36:19.424604 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-29b5c669-f3f1-4060-9355-2ef07bb661cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:36:19.489160 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1294/2585] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:36:19 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:36:19.500745 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd91d80-fc59-491b-b1d4-0c9f2e953899 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:19.502451 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-efd91d80-fc59-491b-b1d4-0c9f2e953899 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:36:19.557588 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd91d80-fc59-491b-b1d4-0c9f2e953899 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:19.561851 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed531d97-cf76-4146-a585-d3f41d95c5a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd05409b38ae346458abfe41ebda1c59e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:19.562477 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed531d97-cf76-4146-a585-d3f41d95c5a3 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:19.562751 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed531d97-cf76-4146-a585-d3f41d95c5a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:19.563020 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed531d97-cf76-4146-a585-d3f41d95c5a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:19.572009 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed531d97-cf76-4146-a585-d3f41d95c5a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:19.573514 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed531d97-cf76-4146-a585-d3f41d95c5a3 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:19.582744 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1292/2586] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:36:19 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:36:19.704493 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b5c669-f3f1-4060-9355-2ef07bb661cd None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:19.763751 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b5c669-f3f1-4060-9355-2ef07bb661cd None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:19.768191 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d8e529-3e5a-4dfa-ae5f-f7b114bf2f8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:19.769031 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d8e529-3e5a-4dfa-ae5f-f7b114bf2f8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:19.769304 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d8e529-3e5a-4dfa-ae5f-f7b114bf2f8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:19.769570 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d8e529-3e5a-4dfa-ae5f-f7b114bf2f8e admin admin] PATH_INFO: `/v3/users/ed7f239fe1e54f2e8d8d29272955079a` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:19.770734 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d8e529-3e5a-4dfa-ae5f-f7b114bf2f8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed7f239fe1e54f2e8d8d29272955079a)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:19.772664 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d8e529-3e5a-4dfa-ae5f-f7b114bf2f8e admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:19.927307 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-b4d8e529-3e5a-4dfa-ae5f-f7b114bf2f8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed7f239fe1e54f2e8d8d29272955079a'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:19.934396 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-b4d8e529-3e5a-4dfa-ae5f-f7b114bf2f8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed7f239fe1e54f2e8d8d29272955079a'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:19.946188 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-b4d8e529-3e5a-4dfa-ae5f-f7b114bf2f8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed7f239fe1e54f2e8d8d29272955079a'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:19.946881 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-b4d8e529-3e5a-4dfa-ae5f-f7b114bf2f8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed7f239fe1e54f2e8d8d29272955079a'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:19.947494 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-b4d8e529-3e5a-4dfa-ae5f-f7b114bf2f8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed7f239fe1e54f2e8d8d29272955079a'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:20.040137 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1295/2587] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 17 19:36:19 2022] DELETE /identity/v3/users/ed7f239fe1e54f2e8d8d29272955079a => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:36:20.045120 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed97f42-00a2-4dbf-86d6-24a00addacb5 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:20.045582 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed97f42-00a2-4dbf-86d6-24a00addacb5 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:20.045684 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed97f42-00a2-4dbf-86d6-24a00addacb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:20.063582 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-3ed97f42-00a2-4dbf-86d6-24a00addacb5 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:36:20.064746 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-3ed97f42-00a2-4dbf-86d6-24a00addacb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:36:20.121581 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1293/2588] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:36:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 17 19:36:20.132432 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d8e529-3e5a-4dfa-ae5f-f7b114bf2f8e admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:20.133815 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-b4d8e529-3e5a-4dfa-ae5f-f7b114bf2f8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:36:20.190023 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d8e529-3e5a-4dfa-ae5f-f7b114bf2f8e admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:20.193680 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a56b88-528b-4b84-8aa7-f0b63cdf5e37 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:20.194389 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a56b88-528b-4b84-8aa7-f0b63cdf5e37 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:20.194657 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a56b88-528b-4b84-8aa7-f0b63cdf5e37 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:20.195023 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a56b88-528b-4b84-8aa7-f0b63cdf5e37 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:20.204099 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a56b88-528b-4b84-8aa7-f0b63cdf5e37 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:20.207596 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a56b88-528b-4b84-8aa7-f0b63cdf5e37 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:20.214744 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1296/2589] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:36:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:36:20.624884 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed97f42-00a2-4dbf-86d6-24a00addacb5 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:20.626321 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-3ed97f42-00a2-4dbf-86d6-24a00addacb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:36:20.638580 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed97f42-00a2-4dbf-86d6-24a00addacb5 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:20.643186 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4d9e9f-0108-4076-b250-b9453afd8a14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:20.644089 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4d9e9f-0108-4076-b250-b9453afd8a14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:20.644368 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4d9e9f-0108-4076-b250-b9453afd8a14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:20.644640 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4d9e9f-0108-4076-b250-b9453afd8a14 admin admin] PATH_INFO: `/v3/projects/b32073e696c740e3995fef607bcc3a92` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:20.645823 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4d9e9f-0108-4076-b250-b9453afd8a14 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b32073e696c740e3995fef607bcc3a92)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:20.647779 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4d9e9f-0108-4076-b250-b9453afd8a14 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:20.856443 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-3e4d9e9f-0108-4076-b250-b9453afd8a14 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b32073e696c740e3995fef607bcc3a92', 'request_id': 'req-3e4d9e9f-0108-4076-b250-b9453afd8a14'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:20.861926 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-3e4d9e9f-0108-4076-b250-b9453afd8a14 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b32073e696c740e3995fef607bcc3a92', 'request_id': 'req-3e4d9e9f-0108-4076-b250-b9453afd8a14'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:20.941060 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-3e4d9e9f-0108-4076-b250-b9453afd8a14 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b32073e696c740e3995fef607bcc3a92', 'request_id': 'req-3e4d9e9f-0108-4076-b250-b9453afd8a14'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:20.953216 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-3e4d9e9f-0108-4076-b250-b9453afd8a14 admin admin] The token cache is being invalidate because project b32073e696c740e3995fef607bcc3a92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 17 19:36:20.953499 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-3e4d9e9f-0108-4076-b250-b9453afd8a14 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:20.954197 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-3e4d9e9f-0108-4076-b250-b9453afd8a14 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:20.955751 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1294/2590] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 17 19:36:20 2022] DELETE /identity/v3/projects/b32073e696c740e3995fef607bcc3a92 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:36:20.959724 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a56b88-528b-4b84-8aa7-f0b63cdf5e37 service neutron] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:20.969785 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a56b88-528b-4b84-8aa7-f0b63cdf5e37 service neutron] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:21.022802 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88edfa87-e05c-449d-9850-ae87bfe11b67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:21.023681 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88edfa87-e05c-449d-9850-ae87bfe11b67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:21.023961 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88edfa87-e05c-449d-9850-ae87bfe11b67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:21.024229 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88edfa87-e05c-449d-9850-ae87bfe11b67 admin admin] PATH_INFO: `/v3/users/683671c03e5b467793e01895bd5501d4` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:21.025394 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88edfa87-e05c-449d-9850-ae87bfe11b67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=683671c03e5b467793e01895bd5501d4)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:21.028003 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88edfa87-e05c-449d-9850-ae87bfe11b67 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:21.211743 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-88edfa87-e05c-449d-9850-ae87bfe11b67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '683671c03e5b467793e01895bd5501d4'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:21.218462 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-88edfa87-e05c-449d-9850-ae87bfe11b67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '683671c03e5b467793e01895bd5501d4'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:21.230161 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-88edfa87-e05c-449d-9850-ae87bfe11b67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '683671c03e5b467793e01895bd5501d4'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:21.231619 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-88edfa87-e05c-449d-9850-ae87bfe11b67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '683671c03e5b467793e01895bd5501d4'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:21.231619 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-88edfa87-e05c-449d-9850-ae87bfe11b67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '683671c03e5b467793e01895bd5501d4'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:21.306526 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1297/2591] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 17 19:36:20 2022] DELETE /identity/v3/users/683671c03e5b467793e01895bd5501d4 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:36:21.658927 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4d9e9f-0108-4076-b250-b9453afd8a14 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:21.660408 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-3e4d9e9f-0108-4076-b250-b9453afd8a14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:36:21.675638 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4d9e9f-0108-4076-b250-b9453afd8a14 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:21.733366 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcd8ee4-13db-4889-8ded-c27bff658aa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:21.734155 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcd8ee4-13db-4889-8ded-c27bff658aa9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:21.734426 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcd8ee4-13db-4889-8ded-c27bff658aa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:21.734691 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcd8ee4-13db-4889-8ded-c27bff658aa9 admin admin] PATH_INFO: `/v3/projects/5862c9f88bbb43559031d065f373a1f6` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:21.735891 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcd8ee4-13db-4889-8ded-c27bff658aa9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5862c9f88bbb43559031d065f373a1f6)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:21.737799 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcd8ee4-13db-4889-8ded-c27bff658aa9 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:21.839102 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-bfcd8ee4-13db-4889-8ded-c27bff658aa9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5862c9f88bbb43559031d065f373a1f6', 'request_id': 'req-bfcd8ee4-13db-4889-8ded-c27bff658aa9'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:21.844316 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-bfcd8ee4-13db-4889-8ded-c27bff658aa9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5862c9f88bbb43559031d065f373a1f6', 'request_id': 'req-bfcd8ee4-13db-4889-8ded-c27bff658aa9'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:21.914951 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-bfcd8ee4-13db-4889-8ded-c27bff658aa9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5862c9f88bbb43559031d065f373a1f6', 'request_id': 'req-bfcd8ee4-13db-4889-8ded-c27bff658aa9'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:21.927016 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-bfcd8ee4-13db-4889-8ded-c27bff658aa9 admin admin] The token cache is being invalidate because project 5862c9f88bbb43559031d065f373a1f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 17 19:36:21.927298 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-bfcd8ee4-13db-4889-8ded-c27bff658aa9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:21.927943 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-bfcd8ee4-13db-4889-8ded-c27bff658aa9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:21.929484 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1295/2592] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 17 19:36:21 2022] DELETE /identity/v3/projects/5862c9f88bbb43559031d065f373a1f6 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:36:21.934355 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0ad91f-9485-4353-8832-0f0c7f6c40d9 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:21.934640 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0ad91f-9485-4353-8832-0f0c7f6c40d9 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:21.934912 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0ad91f-9485-4353-8832-0f0c7f6c40d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:21.952756 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-5c0ad91f-9485-4353-8832-0f0c7f6c40d9 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:36:21.955905 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-5c0ad91f-9485-4353-8832-0f0c7f6c40d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:36:22.012731 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1298/2593] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:36:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:36:22.020396 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6626c294-cc40-4200-b49d-7fc6e72ab65c None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:22.020709 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6626c294-cc40-4200-b49d-7fc6e72ab65c None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:22.020985 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6626c294-cc40-4200-b49d-7fc6e72ab65c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:22.039652 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-6626c294-cc40-4200-b49d-7fc6e72ab65c None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:36:22.040838 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-6626c294-cc40-4200-b49d-7fc6e72ab65c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:36:22.049261 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1296/2594] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:36:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:36:22.053284 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0ad91f-9485-4353-8832-0f0c7f6c40d9 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:22.062563 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0ad91f-9485-4353-8832-0f0c7f6c40d9 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:22.067231 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f3edab-de09-45b1-ae51-8cc3243ce093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:22.067907 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f3edab-de09-45b1-ae51-8cc3243ce093 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:22.068217 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f3edab-de09-45b1-ae51-8cc3243ce093 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:22.068490 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f3edab-de09-45b1-ae51-8cc3243ce093 admin admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:22.069211 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f3edab-de09-45b1-ae51-8cc3243ce093 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:36:22.069514 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f3edab-de09-45b1-ae51-8cc3243ce093 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:22.071126 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f3edab-de09-45b1-ae51-8cc3243ce093 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:22.082488 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1299/2595] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 17 19:36:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:36:22.086562 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6626c294-cc40-4200-b49d-7fc6e72ab65c None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:22.096238 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6626c294-cc40-4200-b49d-7fc6e72ab65c None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:22.100994 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc73edc-082f-4e94-8747-43cb3ec77565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:22.101626 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc73edc-082f-4e94-8747-43cb3ec77565 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:22.101905 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc73edc-082f-4e94-8747-43cb3ec77565 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:22.102177 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc73edc-082f-4e94-8747-43cb3ec77565 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:22.102858 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc73edc-082f-4e94-8747-43cb3ec77565 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:22.105700 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc73edc-082f-4e94-8747-43cb3ec77565 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:22.199434 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1297/2596] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 17 19:36:22 2022] POST /identity/v3/projects => generated 365 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:36:22.203545 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f3edab-de09-45b1-ae51-8cc3243ce093 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:22.213343 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f3edab-de09-45b1-ae51-8cc3243ce093 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:22.267796 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb9e6ad-0c88-4978-8466-60dcfca2434d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:22.268389 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb9e6ad-0c88-4978-8466-60dcfca2434d admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:22.268664 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb9e6ad-0c88-4978-8466-60dcfca2434d admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:22.268933 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb9e6ad-0c88-4978-8466-60dcfca2434d admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:22.269629 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb9e6ad-0c88-4978-8466-60dcfca2434d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:22.271547 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb9e6ad-0c88-4978-8466-60dcfca2434d admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:22.378138 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1300/2597] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 17 19:36:22 2022] POST /identity/v3/users => generated 283 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:36:22.381942 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc73edc-082f-4e94-8747-43cb3ec77565 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:22.392862 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc73edc-082f-4e94-8747-43cb3ec77565 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:22.397666 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b98b488-3bff-40b0-97f3-721e5cf8aa92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:22.398312 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b98b488-3bff-40b0-97f3-721e5cf8aa92 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:22.398589 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b98b488-3bff-40b0-97f3-721e5cf8aa92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:22.398860 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b98b488-3bff-40b0-97f3-721e5cf8aa92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:22.399640 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b98b488-3bff-40b0-97f3-721e5cf8aa92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:22.401351 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b98b488-3bff-40b0-97f3-721e5cf8aa92 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:22.412360 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1298/2598] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:36:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:36:22.416396 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb9e6ad-0c88-4978-8466-60dcfca2434d admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:22.426188 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb9e6ad-0c88-4978-8466-60dcfca2434d admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:22.430758 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be6aa14-23a1-440c-867d-6837e2c901a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:22.431405 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be6aa14-23a1-440c-867d-6837e2c901a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:22.431717 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be6aa14-23a1-440c-867d-6837e2c901a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:22.431992 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be6aa14-23a1-440c-867d-6837e2c901a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:22.432706 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be6aa14-23a1-440c-867d-6837e2c901a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:22.434543 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be6aa14-23a1-440c-867d-6837e2c901a7 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:22.444897 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1301/2599] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:36:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:36:22.449197 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b98b488-3bff-40b0-97f3-721e5cf8aa92 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:22.458758 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b98b488-3bff-40b0-97f3-721e5cf8aa92 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:22.463364 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c14886-5302-4b4f-be0e-ab2b1a70e87d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:22.464917 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c14886-5302-4b4f-be0e-ab2b1a70e87d admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:22.465194 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c14886-5302-4b4f-be0e-ab2b1a70e87d admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:22.465462 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c14886-5302-4b4f-be0e-ab2b1a70e87d admin admin] PATH_INFO: `/v3/projects/d9869c7deb1b486eb58766f2dbf5c045/users/7e5bfa425fe24214b0ad5844beb7dd2a/roles/114e6a16e63447d1924707ace5aace25` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:22.479796 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c14886-5302-4b4f-be0e-ab2b1a70e87d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9869c7deb1b486eb58766f2dbf5c045, user_id=7e5bfa425fe24214b0ad5844beb7dd2a, role_id=114e6a16e63447d1924707ace5aace25)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:22.482452 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c14886-5302-4b4f-be0e-ab2b1a70e87d admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:22.547130 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1299/2600] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:36:22 2022] PUT /identity/v3/projects/d9869c7deb1b486eb58766f2dbf5c045/users/7e5bfa425fe24214b0ad5844beb7dd2a/roles/114e6a16e63447d1924707ace5aace25 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:36:22.557642 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae359a9-f098-4dc3-84fa-5bdf2f1992bf None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:22.558024 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae359a9-f098-4dc3-84fa-5bdf2f1992bf None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:22.558306 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae359a9-f098-4dc3-84fa-5bdf2f1992bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:22.584397 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-eae359a9-f098-4dc3-84fa-5bdf2f1992bf None None] MFA Rules not processed for user `7e5bfa425fe24214b0ad5844beb7dd2a`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:36:22.585856 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-eae359a9-f098-4dc3-84fa-5bdf2f1992bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:36:22.647269 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1302/2601] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:36:22 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 17 19:36:22.652249 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c14886-5302-4b4f-be0e-ab2b1a70e87d admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:22.711089 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c14886-5302-4b4f-be0e-ab2b1a70e87d admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:22.716302 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c9b20a-8e0b-45a3-a410-52b667084df2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:22.716945 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c9b20a-8e0b-45a3-a410-52b667084df2 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:22.717223 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c9b20a-8e0b-45a3-a410-52b667084df2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:22.717490 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c9b20a-8e0b-45a3-a410-52b667084df2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:22.718159 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c9b20a-8e0b-45a3-a410-52b667084df2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:22.720092 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c9b20a-8e0b-45a3-a410-52b667084df2 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:22.828030 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1300/2602] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 17 19:36:22 2022] POST /identity/v3/projects => generated 367 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:36:22.831979 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae359a9-f098-4dc3-84fa-5bdf2f1992bf None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:22.843067 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae359a9-f098-4dc3-84fa-5bdf2f1992bf None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:22.897705 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83779861-6cd9-4435-b9d2-361200c47003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:22.898360 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83779861-6cd9-4435-b9d2-361200c47003 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:22.898637 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83779861-6cd9-4435-b9d2-361200c47003 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:22.898908 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83779861-6cd9-4435-b9d2-361200c47003 admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:22.899699 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83779861-6cd9-4435-b9d2-361200c47003 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:22.901585 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83779861-6cd9-4435-b9d2-361200c47003 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:22.977424 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1303/2603] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 17 19:36:22 2022] POST /identity/v3/users => generated 290 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:36:22.982306 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c9b20a-8e0b-45a3-a410-52b667084df2 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:22.992612 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c9b20a-8e0b-45a3-a410-52b667084df2 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:22.997504 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a06bd5-a8aa-4d78-9bfd-e38339ebc914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:22.998152 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a06bd5-a8aa-4d78-9bfd-e38339ebc914 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:22.998439 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a06bd5-a8aa-4d78-9bfd-e38339ebc914 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:22.998717 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a06bd5-a8aa-4d78-9bfd-e38339ebc914 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:22.999450 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a06bd5-a8aa-4d78-9bfd-e38339ebc914 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:23.001074 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a06bd5-a8aa-4d78-9bfd-e38339ebc914 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:23.012036 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1301/2604] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:36:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:36:23.016032 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83779861-6cd9-4435-b9d2-361200c47003 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:23.025858 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83779861-6cd9-4435-b9d2-361200c47003 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:23.031084 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e737fbf1-8c5f-4e57-ab79-59aa101fd33d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:23.032008 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e737fbf1-8c5f-4e57-ab79-59aa101fd33d admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:23.032287 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e737fbf1-8c5f-4e57-ab79-59aa101fd33d admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:23.032559 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e737fbf1-8c5f-4e57-ab79-59aa101fd33d admin admin] PATH_INFO: `/v3/projects/8e642378b6304c8d9b2d921d0a512307/users/54e11d5939c948aa9f30a50cb7cf701e/roles/60af5088793e4379b88821a86552a344` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:23.046791 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e737fbf1-8c5f-4e57-ab79-59aa101fd33d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e642378b6304c8d9b2d921d0a512307, user_id=54e11d5939c948aa9f30a50cb7cf701e, role_id=60af5088793e4379b88821a86552a344)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:23.049479 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e737fbf1-8c5f-4e57-ab79-59aa101fd33d admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:23.146772 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1304/2605] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:36:23 2022] PUT /identity/v3/projects/8e642378b6304c8d9b2d921d0a512307/users/54e11d5939c948aa9f30a50cb7cf701e/roles/60af5088793e4379b88821a86552a344 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 17 19:36:23.150678 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a06bd5-a8aa-4d78-9bfd-e38339ebc914 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:23.160919 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a06bd5-a8aa-4d78-9bfd-e38339ebc914 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:23.212833 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e38bf7-3e0f-42f3-a6ea-702d32815041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:23.213478 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e38bf7-3e0f-42f3-a6ea-702d32815041 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:23.213753 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e38bf7-3e0f-42f3-a6ea-702d32815041 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:23.214020 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e38bf7-3e0f-42f3-a6ea-702d32815041 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:23.215341 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e38bf7-3e0f-42f3-a6ea-702d32815041 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:23.216966 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e38bf7-3e0f-42f3-a6ea-702d32815041 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:23.226117 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1302/2606] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:36:23 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:36:23.230391 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e737fbf1-8c5f-4e57-ab79-59aa101fd33d admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:23.240600 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e737fbf1-8c5f-4e57-ab79-59aa101fd33d admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:23.245232 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7650f050-ddd4-4d52-9fc4-29315fd6dfb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:23.246092 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7650f050-ddd4-4d52-9fc4-29315fd6dfb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:23.246369 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7650f050-ddd4-4d52-9fc4-29315fd6dfb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:23.246642 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7650f050-ddd4-4d52-9fc4-29315fd6dfb1 admin admin] PATH_INFO: `/v3/projects/8e642378b6304c8d9b2d921d0a512307/users/54e11d5939c948aa9f30a50cb7cf701e/roles/60af5088793e4379b88821a86552a344` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:23.248628 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7650f050-ddd4-4d52-9fc4-29315fd6dfb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e642378b6304c8d9b2d921d0a512307, user_id=54e11d5939c948aa9f30a50cb7cf701e, role_id=60af5088793e4379b88821a86552a344)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:23.251252 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7650f050-ddd4-4d52-9fc4-29315fd6dfb1 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:23.261041 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1305/2607] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:36:23 2022] PUT /identity/v3/projects/8e642378b6304c8d9b2d921d0a512307/users/54e11d5939c948aa9f30a50cb7cf701e/roles/60af5088793e4379b88821a86552a344 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 17 19:36:23.272154 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9eea16-7aa7-4c09-a2ec-787aa962acbc None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:23.272442 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9eea16-7aa7-4c09-a2ec-787aa962acbc None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:23.272714 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9eea16-7aa7-4c09-a2ec-787aa962acbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:23.298452 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-ed9eea16-7aa7-4c09-a2ec-787aa962acbc None None] MFA Rules not processed for user `54e11d5939c948aa9f30a50cb7cf701e`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:36:23.299665 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-ed9eea16-7aa7-4c09-a2ec-787aa962acbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:36:23.364844 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1303/2608] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:36:23 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:36:23.377381 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7650f050-ddd4-4d52-9fc4-29315fd6dfb1 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:23.379069 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-7650f050-ddd4-4d52-9fc4-29315fd6dfb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:36:23.435867 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7650f050-ddd4-4d52-9fc4-29315fd6dfb1 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:23.439481 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbf3967-883c-4920-a1b2-a4ade92e5d5e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd05409b38ae346458abfe41ebda1c59e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:23.440140 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbf3967-883c-4920-a1b2-a4ade92e5d5e service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:23.440418 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbf3967-883c-4920-a1b2-a4ade92e5d5e service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:23.440687 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbf3967-883c-4920-a1b2-a4ade92e5d5e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:23.449999 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbf3967-883c-4920-a1b2-a4ade92e5d5e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:23.451404 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbf3967-883c-4920-a1b2-a4ade92e5d5e service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:23.460612 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1306/2609] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:36:23 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:36:23.684865 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9eea16-7aa7-4c09-a2ec-787aa962acbc None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:23.694286 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9eea16-7aa7-4c09-a2ec-787aa962acbc None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:23.697877 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acf6deb-3259-4cd2-af00-8f08140af805 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd05409b38ae346458abfe41ebda1c59e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:23.699553 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acf6deb-3259-4cd2-af00-8f08140af805 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:23.699553 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acf6deb-3259-4cd2-af00-8f08140af805 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:23.699845 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acf6deb-3259-4cd2-af00-8f08140af805 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:23.755430 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acf6deb-3259-4cd2-af00-8f08140af805 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:23.756897 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acf6deb-3259-4cd2-af00-8f08140af805 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:23.766164 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1304/2610] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:36:23 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:36:29.905551 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbf3967-883c-4920-a1b2-a4ade92e5d5e service cinder] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:29.964477 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbf3967-883c-4920-a1b2-a4ade92e5d5e service cinder] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:29.969550 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5131d8-ad35-4868-b7a3-6196a9743908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:29.970397 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5131d8-ad35-4868-b7a3-6196a9743908 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:29.970673 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5131d8-ad35-4868-b7a3-6196a9743908 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:29.970942 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5131d8-ad35-4868-b7a3-6196a9743908 admin admin] PATH_INFO: `/v3/users/7e5bfa425fe24214b0ad5844beb7dd2a` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:29.972113 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5131d8-ad35-4868-b7a3-6196a9743908 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e5bfa425fe24214b0ad5844beb7dd2a)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:29.974006 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5131d8-ad35-4868-b7a3-6196a9743908 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:30.171220 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-9c5131d8-ad35-4868-b7a3-6196a9743908 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e5bfa425fe24214b0ad5844beb7dd2a'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:30.177767 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-9c5131d8-ad35-4868-b7a3-6196a9743908 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e5bfa425fe24214b0ad5844beb7dd2a'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:30.189899 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-9c5131d8-ad35-4868-b7a3-6196a9743908 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e5bfa425fe24214b0ad5844beb7dd2a'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:30.190560 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-9c5131d8-ad35-4868-b7a3-6196a9743908 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e5bfa425fe24214b0ad5844beb7dd2a'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:30.191151 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-9c5131d8-ad35-4868-b7a3-6196a9743908 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e5bfa425fe24214b0ad5844beb7dd2a'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:30.260699 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1307/2611] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 17 19:36:29 2022] DELETE /identity/v3/users/7e5bfa425fe24214b0ad5844beb7dd2a => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:36:30.265790 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07465d20-8504-47ac-ac7f-2e991f9d4736 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:30.266271 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07465d20-8504-47ac-ac7f-2e991f9d4736 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:30.266382 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07465d20-8504-47ac-ac7f-2e991f9d4736 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:30.283071 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-07465d20-8504-47ac-ac7f-2e991f9d4736 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:36:30.284636 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-07465d20-8504-47ac-ac7f-2e991f9d4736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:36:30.341068 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1305/2612] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:36:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:36:30.350639 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5131d8-ad35-4868-b7a3-6196a9743908 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:30.352052 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-9c5131d8-ad35-4868-b7a3-6196a9743908 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:36:30.406666 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5131d8-ad35-4868-b7a3-6196a9743908 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:30.410293 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a77d77-a41a-45dd-b800-56b97d396384 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:30.410921 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a77d77-a41a-45dd-b800-56b97d396384 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:30.411200 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a77d77-a41a-45dd-b800-56b97d396384 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:30.411468 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a77d77-a41a-45dd-b800-56b97d396384 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:30.420192 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a77d77-a41a-45dd-b800-56b97d396384 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:30.421652 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a77d77-a41a-45dd-b800-56b97d396384 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:30.430895 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1308/2613] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:36:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:36:30.824782 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07465d20-8504-47ac-ac7f-2e991f9d4736 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:30.826227 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-07465d20-8504-47ac-ac7f-2e991f9d4736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:36:30.838403 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07465d20-8504-47ac-ac7f-2e991f9d4736 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:30.843753 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee53e15-6b7f-424e-a501-b479ee817a94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:30.844675 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee53e15-6b7f-424e-a501-b479ee817a94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:30.844985 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee53e15-6b7f-424e-a501-b479ee817a94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:30.845266 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee53e15-6b7f-424e-a501-b479ee817a94 admin admin] PATH_INFO: `/v3/projects/d9869c7deb1b486eb58766f2dbf5c045` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:30.846458 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee53e15-6b7f-424e-a501-b479ee817a94 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9869c7deb1b486eb58766f2dbf5c045)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:30.848440 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee53e15-6b7f-424e-a501-b479ee817a94 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:30.989092 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-6ee53e15-6b7f-424e-a501-b479ee817a94 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9869c7deb1b486eb58766f2dbf5c045', 'request_id': 'req-6ee53e15-6b7f-424e-a501-b479ee817a94'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:30.994374 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-6ee53e15-6b7f-424e-a501-b479ee817a94 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9869c7deb1b486eb58766f2dbf5c045', 'request_id': 'req-6ee53e15-6b7f-424e-a501-b479ee817a94'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:31.062649 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-6ee53e15-6b7f-424e-a501-b479ee817a94 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9869c7deb1b486eb58766f2dbf5c045', 'request_id': 'req-6ee53e15-6b7f-424e-a501-b479ee817a94'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:31.073405 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-6ee53e15-6b7f-424e-a501-b479ee817a94 admin admin] The token cache is being invalidate because project d9869c7deb1b486eb58766f2dbf5c045 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 17 19:36:31.073687 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-6ee53e15-6b7f-424e-a501-b479ee817a94 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:31.074343 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-6ee53e15-6b7f-424e-a501-b479ee817a94 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:31.076246 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1306/2614] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 17 19:36:30 2022] DELETE /identity/v3/projects/d9869c7deb1b486eb58766f2dbf5c045 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:36:31.080208 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a77d77-a41a-45dd-b800-56b97d396384 service neutron] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:31.090876 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a77d77-a41a-45dd-b800-56b97d396384 service neutron] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:31.143398 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad612f0-e35c-4ba1-8a22-d7cfc553ba1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:31.144272 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad612f0-e35c-4ba1-8a22-d7cfc553ba1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:31.144542 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad612f0-e35c-4ba1-8a22-d7cfc553ba1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:31.144806 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad612f0-e35c-4ba1-8a22-d7cfc553ba1d admin admin] PATH_INFO: `/v3/users/54e11d5939c948aa9f30a50cb7cf701e` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:31.145952 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad612f0-e35c-4ba1-8a22-d7cfc553ba1d admin admin] RBAC: Authorizing `identity:delete_user(user_id=54e11d5939c948aa9f30a50cb7cf701e)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:31.147958 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad612f0-e35c-4ba1-8a22-d7cfc553ba1d admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:31.298372 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-5ad612f0-e35c-4ba1-8a22-d7cfc553ba1d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54e11d5939c948aa9f30a50cb7cf701e'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:31.304847 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-5ad612f0-e35c-4ba1-8a22-d7cfc553ba1d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54e11d5939c948aa9f30a50cb7cf701e'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:31.316206 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-5ad612f0-e35c-4ba1-8a22-d7cfc553ba1d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54e11d5939c948aa9f30a50cb7cf701e'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:31.316861 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-5ad612f0-e35c-4ba1-8a22-d7cfc553ba1d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54e11d5939c948aa9f30a50cb7cf701e'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:31.317458 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-5ad612f0-e35c-4ba1-8a22-d7cfc553ba1d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54e11d5939c948aa9f30a50cb7cf701e'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:31.382682 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1309/2615] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 17 19:36:31 2022] DELETE /identity/v3/users/54e11d5939c948aa9f30a50cb7cf701e => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:36:31.778994 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee53e15-6b7f-424e-a501-b479ee817a94 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:31.780499 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-6ee53e15-6b7f-424e-a501-b479ee817a94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:36:31.795162 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee53e15-6b7f-424e-a501-b479ee817a94 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:31.851671 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686cc11d-5dcb-4a8a-965e-075b073eb21c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:31.852454 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686cc11d-5dcb-4a8a-965e-075b073eb21c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:31.852724 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686cc11d-5dcb-4a8a-965e-075b073eb21c admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:31.852988 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686cc11d-5dcb-4a8a-965e-075b073eb21c admin admin] PATH_INFO: `/v3/projects/8e642378b6304c8d9b2d921d0a512307` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:31.854391 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686cc11d-5dcb-4a8a-965e-075b073eb21c admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e642378b6304c8d9b2d921d0a512307)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:31.856321 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686cc11d-5dcb-4a8a-965e-075b073eb21c admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:31.959880 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-686cc11d-5dcb-4a8a-965e-075b073eb21c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e642378b6304c8d9b2d921d0a512307', 'request_id': 'req-686cc11d-5dcb-4a8a-965e-075b073eb21c'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:31.965222 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-686cc11d-5dcb-4a8a-965e-075b073eb21c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e642378b6304c8d9b2d921d0a512307', 'request_id': 'req-686cc11d-5dcb-4a8a-965e-075b073eb21c'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:32.022848 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-686cc11d-5dcb-4a8a-965e-075b073eb21c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e642378b6304c8d9b2d921d0a512307', 'request_id': 'req-686cc11d-5dcb-4a8a-965e-075b073eb21c'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:32.034045 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-686cc11d-5dcb-4a8a-965e-075b073eb21c admin admin] The token cache is being invalidate because project 8e642378b6304c8d9b2d921d0a512307 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 17 19:36:32.034334 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-686cc11d-5dcb-4a8a-965e-075b073eb21c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:32.034954 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-686cc11d-5dcb-4a8a-965e-075b073eb21c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:32.036516 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1307/2616] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 17 19:36:31 2022] DELETE /identity/v3/projects/8e642378b6304c8d9b2d921d0a512307 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:36:32.041462 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c231769c-1722-4aa2-8ca3-5dd2d20f3d29 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:32.041746 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c231769c-1722-4aa2-8ca3-5dd2d20f3d29 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:32.042016 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c231769c-1722-4aa2-8ca3-5dd2d20f3d29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:32.059904 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-c231769c-1722-4aa2-8ca3-5dd2d20f3d29 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:36:32.061083 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-c231769c-1722-4aa2-8ca3-5dd2d20f3d29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:36:32.117167 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1310/2617] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:36:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 17 19:36:32.125283 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d486d62-959f-4f7f-b2d5-b467c3d59d55 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:32.125566 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d486d62-959f-4f7f-b2d5-b467c3d59d55 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:32.125838 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d486d62-959f-4f7f-b2d5-b467c3d59d55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:32.143223 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-9d486d62-959f-4f7f-b2d5-b467c3d59d55 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:36:32.144429 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-9d486d62-959f-4f7f-b2d5-b467c3d59d55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:36:32.152078 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1308/2618] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:36:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:36:32.155914 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c231769c-1722-4aa2-8ca3-5dd2d20f3d29 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:32.165231 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c231769c-1722-4aa2-8ca3-5dd2d20f3d29 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:32.170336 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c677f46-12a6-4ca5-aa5a-f5c5f0a07560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:32.170967 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c677f46-12a6-4ca5-aa5a-f5c5f0a07560 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:32.171245 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c677f46-12a6-4ca5-aa5a-f5c5f0a07560 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:32.171547 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c677f46-12a6-4ca5-aa5a-f5c5f0a07560 admin admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:32.172268 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c677f46-12a6-4ca5-aa5a-f5c5f0a07560 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:36:32.172566 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c677f46-12a6-4ca5-aa5a-f5c5f0a07560 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:32.174142 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c677f46-12a6-4ca5-aa5a-f5c5f0a07560 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:32.185883 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1311/2619] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 17 19:36:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:36:32.189962 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d486d62-959f-4f7f-b2d5-b467c3d59d55 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:32.198726 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d486d62-959f-4f7f-b2d5-b467c3d59d55 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:32.203289 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debf8a0a-735c-435e-9c08-e4b3142b6f3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:32.203948 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debf8a0a-735c-435e-9c08-e4b3142b6f3b admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:32.204223 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debf8a0a-735c-435e-9c08-e4b3142b6f3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:32.204493 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debf8a0a-735c-435e-9c08-e4b3142b6f3b admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:32.205166 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debf8a0a-735c-435e-9c08-e4b3142b6f3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:32.207104 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debf8a0a-735c-435e-9c08-e4b3142b6f3b admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:32.292323 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1309/2620] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 17 19:36:32 2022] POST /identity/v3/projects => generated 375 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 17 19:36:32.296260 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c677f46-12a6-4ca5-aa5a-f5c5f0a07560 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:32.306061 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c677f46-12a6-4ca5-aa5a-f5c5f0a07560 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:32.360434 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceddde3-7242-4be7-b1ac-c6ea2c089cf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:32.361108 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceddde3-7242-4be7-b1ac-c6ea2c089cf2 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:32.361369 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceddde3-7242-4be7-b1ac-c6ea2c089cf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:32.361641 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceddde3-7242-4be7-b1ac-c6ea2c089cf2 admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:32.362321 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aceddde3-7242-4be7-b1ac-c6ea2c089cf2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:32.364254 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aceddde3-7242-4be7-b1ac-c6ea2c089cf2 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:32.466835 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1312/2621] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 17 19:36:32 2022] POST /identity/v3/users => generated 288 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:36:32.470785 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debf8a0a-735c-435e-9c08-e4b3142b6f3b admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:32.481181 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debf8a0a-735c-435e-9c08-e4b3142b6f3b admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:32.486059 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab95330-9f49-4753-8b10-62283fe5aaeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:32.486709 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab95330-9f49-4753-8b10-62283fe5aaeb admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:32.486984 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab95330-9f49-4753-8b10-62283fe5aaeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:32.487253 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab95330-9f49-4753-8b10-62283fe5aaeb admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:32.487991 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab95330-9f49-4753-8b10-62283fe5aaeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:32.489681 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab95330-9f49-4753-8b10-62283fe5aaeb admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:32.500561 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1310/2622] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:36:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:36:32.504546 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceddde3-7242-4be7-b1ac-c6ea2c089cf2 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:32.515183 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceddde3-7242-4be7-b1ac-c6ea2c089cf2 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:32.520038 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de39164f-7193-4eaf-a73d-691634281a55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:32.520682 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de39164f-7193-4eaf-a73d-691634281a55 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:32.520961 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de39164f-7193-4eaf-a73d-691634281a55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:32.521232 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de39164f-7193-4eaf-a73d-691634281a55 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:32.521944 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de39164f-7193-4eaf-a73d-691634281a55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:32.523645 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de39164f-7193-4eaf-a73d-691634281a55 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:32.533181 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1313/2623] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:36:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:36:32.537019 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab95330-9f49-4753-8b10-62283fe5aaeb admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:32.547060 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab95330-9f49-4753-8b10-62283fe5aaeb admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:32.551724 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e651b1-2793-4d17-96e9-eb8458a8d640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:32.552587 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e651b1-2793-4d17-96e9-eb8458a8d640 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:32.552864 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e651b1-2793-4d17-96e9-eb8458a8d640 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:32.553136 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e651b1-2793-4d17-96e9-eb8458a8d640 admin admin] PATH_INFO: `/v3/projects/f7b05cc7a00a4971945f7e89ce0a8878/users/ba8512a78bd04527a91fc6cdada51aa3/roles/114e6a16e63447d1924707ace5aace25` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:32.567633 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e651b1-2793-4d17-96e9-eb8458a8d640 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7b05cc7a00a4971945f7e89ce0a8878, user_id=ba8512a78bd04527a91fc6cdada51aa3, role_id=114e6a16e63447d1924707ace5aace25)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:32.570276 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e651b1-2793-4d17-96e9-eb8458a8d640 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:32.638505 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1311/2624] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:36:32 2022] PUT /identity/v3/projects/f7b05cc7a00a4971945f7e89ce0a8878/users/ba8512a78bd04527a91fc6cdada51aa3/roles/114e6a16e63447d1924707ace5aace25 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:36:32.648901 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ea8b69-b075-4d3b-802f-0c45a8b31fe6 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:32.649194 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ea8b69-b075-4d3b-802f-0c45a8b31fe6 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:32.649471 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ea8b69-b075-4d3b-802f-0c45a8b31fe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:32.674649 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-53ea8b69-b075-4d3b-802f-0c45a8b31fe6 None None] MFA Rules not processed for user `ba8512a78bd04527a91fc6cdada51aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:36:32.675878 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-53ea8b69-b075-4d3b-802f-0c45a8b31fe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:36:32.737064 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1314/2625] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:36:32 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:36:32.741404 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e651b1-2793-4d17-96e9-eb8458a8d640 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:32.799447 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e651b1-2793-4d17-96e9-eb8458a8d640 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:32.804215 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb07509b-c84f-498d-90e8-2a19eb0fd080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:32.804882 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb07509b-c84f-498d-90e8-2a19eb0fd080 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:32.805231 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb07509b-c84f-498d-90e8-2a19eb0fd080 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:32.805631 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb07509b-c84f-498d-90e8-2a19eb0fd080 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:32.806384 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb07509b-c84f-498d-90e8-2a19eb0fd080 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:32.808512 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb07509b-c84f-498d-90e8-2a19eb0fd080 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:32.896984 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1312/2626] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 17 19:36:32 2022] POST /identity/v3/projects => generated 377 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:36:32.901031 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ea8b69-b075-4d3b-802f-0c45a8b31fe6 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:32.911434 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ea8b69-b075-4d3b-802f-0c45a8b31fe6 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:32.965296 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7abae60-c19a-444d-88ec-850371354007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:32.965958 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7abae60-c19a-444d-88ec-850371354007 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:32.966232 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7abae60-c19a-444d-88ec-850371354007 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:32.966498 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7abae60-c19a-444d-88ec-850371354007 admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:32.967165 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7abae60-c19a-444d-88ec-850371354007 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:32.969082 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7abae60-c19a-444d-88ec-850371354007 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:33.076002 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1315/2627] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 17 19:36:32 2022] POST /identity/v3/users => generated 289 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:36:33.077403 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb07509b-c84f-498d-90e8-2a19eb0fd080 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:33.087340 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb07509b-c84f-498d-90e8-2a19eb0fd080 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:33.094509 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2249fd7b-627d-4773-8840-bb7885f2d892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:33.095158 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2249fd7b-627d-4773-8840-bb7885f2d892 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:33.095433 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2249fd7b-627d-4773-8840-bb7885f2d892 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:33.095732 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2249fd7b-627d-4773-8840-bb7885f2d892 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:33.096479 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2249fd7b-627d-4773-8840-bb7885f2d892 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:33.098057 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2249fd7b-627d-4773-8840-bb7885f2d892 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:33.112292 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1313/2628] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:36:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:36:33.113527 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7abae60-c19a-444d-88ec-850371354007 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:33.123433 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7abae60-c19a-444d-88ec-850371354007 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:33.128188 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af16a575-dbba-4df2-8e49-c574c0390489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:33.128839 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af16a575-dbba-4df2-8e49-c574c0390489 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:33.129118 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af16a575-dbba-4df2-8e49-c574c0390489 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:33.129467 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af16a575-dbba-4df2-8e49-c574c0390489 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:33.130170 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af16a575-dbba-4df2-8e49-c574c0390489 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:33.131780 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af16a575-dbba-4df2-8e49-c574c0390489 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:33.141802 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1316/2629] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:36:33 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:36:33.145880 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2249fd7b-627d-4773-8840-bb7885f2d892 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:33.156714 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2249fd7b-627d-4773-8840-bb7885f2d892 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:33.161585 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df342133-783c-49c5-a21d-e067dd7d6199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:33.162463 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df342133-783c-49c5-a21d-e067dd7d6199 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:33.162742 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df342133-783c-49c5-a21d-e067dd7d6199 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:33.163014 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df342133-783c-49c5-a21d-e067dd7d6199 admin admin] PATH_INFO: `/v3/projects/be9bfffcacb04ff3a2c1e8677166ca30/users/7c086461ecf14627bcaf0a7a9498fa93/roles/114e6a16e63447d1924707ace5aace25` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:33.178415 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df342133-783c-49c5-a21d-e067dd7d6199 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be9bfffcacb04ff3a2c1e8677166ca30, user_id=7c086461ecf14627bcaf0a7a9498fa93, role_id=114e6a16e63447d1924707ace5aace25)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:33.181095 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df342133-783c-49c5-a21d-e067dd7d6199 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:33.261238 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1314/2630] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:36:33 2022] PUT /identity/v3/projects/be9bfffcacb04ff3a2c1e8677166ca30/users/7c086461ecf14627bcaf0a7a9498fa93/roles/114e6a16e63447d1924707ace5aace25 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:36:33.323558 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61752a67-2896-4d56-a46a-0c9f2f06fec7 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:33.323785 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61752a67-2896-4d56-a46a-0c9f2f06fec7 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:33.324072 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61752a67-2896-4d56-a46a-0c9f2f06fec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:33.350385 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-61752a67-2896-4d56-a46a-0c9f2f06fec7 None None] MFA Rules not processed for user `7c086461ecf14627bcaf0a7a9498fa93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:36:33.351612 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-61752a67-2896-4d56-a46a-0c9f2f06fec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:36:33.412719 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1317/2631] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:36:33 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:36:33.417289 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df342133-783c-49c5-a21d-e067dd7d6199 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:33.477712 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df342133-783c-49c5-a21d-e067dd7d6199 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:33.484351 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6733cb38-bffe-40c9-8633-560fd5842b0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:33.485167 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6733cb38-bffe-40c9-8633-560fd5842b0e admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:33.485443 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6733cb38-bffe-40c9-8633-560fd5842b0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:33.485808 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6733cb38-bffe-40c9-8633-560fd5842b0e admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:33.486534 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6733cb38-bffe-40c9-8633-560fd5842b0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:33.488506 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6733cb38-bffe-40c9-8633-560fd5842b0e admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:33.639822 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1315/2632] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 17 19:36:33 2022] POST /identity/v3/projects => generated 375 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:36:33.640954 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61752a67-2896-4d56-a46a-0c9f2f06fec7 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:33.651467 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61752a67-2896-4d56-a46a-0c9f2f06fec7 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:33.705510 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0734b4a4-b171-4b22-9aa9-cef54e6e9a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:33.706169 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0734b4a4-b171-4b22-9aa9-cef54e6e9a2a admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:33.706450 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0734b4a4-b171-4b22-9aa9-cef54e6e9a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:33.706723 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0734b4a4-b171-4b22-9aa9-cef54e6e9a2a admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:33.707406 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0734b4a4-b171-4b22-9aa9-cef54e6e9a2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:33.709396 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0734b4a4-b171-4b22-9aa9-cef54e6e9a2a admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:33.783973 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1318/2633] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 17 19:36:33 2022] POST /identity/v3/users => generated 294 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 17 19:36:33.787889 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6733cb38-bffe-40c9-8633-560fd5842b0e admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:33.798207 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6733cb38-bffe-40c9-8633-560fd5842b0e admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:33.802823 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a49e5c-60ed-4ffa-b639-1ecad3adb66a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:33.803601 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a49e5c-60ed-4ffa-b639-1ecad3adb66a admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:33.806006 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a49e5c-60ed-4ffa-b639-1ecad3adb66a admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:33.806006 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a49e5c-60ed-4ffa-b639-1ecad3adb66a admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:33.806006 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a49e5c-60ed-4ffa-b639-1ecad3adb66a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:33.806472 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a49e5c-60ed-4ffa-b639-1ecad3adb66a admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:33.816938 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1316/2634] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:36:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:36:33.820866 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0734b4a4-b171-4b22-9aa9-cef54e6e9a2a admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:33.831088 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0734b4a4-b171-4b22-9aa9-cef54e6e9a2a admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:33.835570 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958b677a-8baa-4cd2-bfc4-45fcb90e493e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:33.836422 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958b677a-8baa-4cd2-bfc4-45fcb90e493e admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:33.836695 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958b677a-8baa-4cd2-bfc4-45fcb90e493e admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:33.837046 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958b677a-8baa-4cd2-bfc4-45fcb90e493e admin admin] PATH_INFO: `/v3/projects/d655823fda984fc78354814f5e7f7580/users/8a329069307e451aafd8caee5741214a/roles/60af5088793e4379b88821a86552a344` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:33.853373 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958b677a-8baa-4cd2-bfc4-45fcb90e493e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d655823fda984fc78354814f5e7f7580, user_id=8a329069307e451aafd8caee5741214a, role_id=60af5088793e4379b88821a86552a344)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:33.856318 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958b677a-8baa-4cd2-bfc4-45fcb90e493e admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:33.914564 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1319/2635] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:36:33 2022] PUT /identity/v3/projects/d655823fda984fc78354814f5e7f7580/users/8a329069307e451aafd8caee5741214a/roles/60af5088793e4379b88821a86552a344 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:36:33.918401 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a49e5c-60ed-4ffa-b639-1ecad3adb66a admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:33.928391 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a49e5c-60ed-4ffa-b639-1ecad3adb66a admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:33.981797 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6694cfd-5acb-4758-b49d-262d6bcdeac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:33.982446 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6694cfd-5acb-4758-b49d-262d6bcdeac2 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:33.982722 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6694cfd-5acb-4758-b49d-262d6bcdeac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:33.982990 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6694cfd-5acb-4758-b49d-262d6bcdeac2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:33.983724 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6694cfd-5acb-4758-b49d-262d6bcdeac2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:33.985336 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6694cfd-5acb-4758-b49d-262d6bcdeac2 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:33.995189 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1317/2636] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:36:33 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:36:33.999797 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958b677a-8baa-4cd2-bfc4-45fcb90e493e admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:34.009958 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958b677a-8baa-4cd2-bfc4-45fcb90e493e admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:34.015215 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760f50e8-d180-4ab9-ae00-66e4c6a83114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:34.016115 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760f50e8-d180-4ab9-ae00-66e4c6a83114 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:34.016396 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760f50e8-d180-4ab9-ae00-66e4c6a83114 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:34.016697 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760f50e8-d180-4ab9-ae00-66e4c6a83114 admin admin] PATH_INFO: `/v3/projects/d655823fda984fc78354814f5e7f7580/users/8a329069307e451aafd8caee5741214a/roles/60af5088793e4379b88821a86552a344` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:34.018644 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760f50e8-d180-4ab9-ae00-66e4c6a83114 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d655823fda984fc78354814f5e7f7580, user_id=8a329069307e451aafd8caee5741214a, role_id=60af5088793e4379b88821a86552a344)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:34.021299 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760f50e8-d180-4ab9-ae00-66e4c6a83114 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:34.031617 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1320/2637] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:36:33 2022] PUT /identity/v3/projects/d655823fda984fc78354814f5e7f7580/users/8a329069307e451aafd8caee5741214a/roles/60af5088793e4379b88821a86552a344 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:36:34.041550 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98feb006-9770-4513-8bee-4d87be2997f0 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:34.041838 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98feb006-9770-4513-8bee-4d87be2997f0 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:34.042114 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98feb006-9770-4513-8bee-4d87be2997f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:34.068823 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-98feb006-9770-4513-8bee-4d87be2997f0 None None] MFA Rules not processed for user `8a329069307e451aafd8caee5741214a`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:36:34.070015 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-98feb006-9770-4513-8bee-4d87be2997f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:36:34.134612 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1318/2638] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:36:34 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:36:34.146391 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760f50e8-d180-4ab9-ae00-66e4c6a83114 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:34.148065 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-760f50e8-d180-4ab9-ae00-66e4c6a83114 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:36:34.203130 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760f50e8-d180-4ab9-ae00-66e4c6a83114 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:34.206762 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0394176-682e-4d7a-a1be-5a5fb987396f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd05409b38ae346458abfe41ebda1c59e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:34.207386 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0394176-682e-4d7a-a1be-5a5fb987396f service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:34.207695 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0394176-682e-4d7a-a1be-5a5fb987396f service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:34.207971 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0394176-682e-4d7a-a1be-5a5fb987396f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:34.261516 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0394176-682e-4d7a-a1be-5a5fb987396f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:34.263428 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0394176-682e-4d7a-a1be-5a5fb987396f service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:34.271910 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1321/2639] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:36:34 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:36:36.326459 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98feb006-9770-4513-8bee-4d87be2997f0 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:36.328024 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-98feb006-9770-4513-8bee-4d87be2997f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:36:36.340983 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98feb006-9770-4513-8bee-4d87be2997f0 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:36.345529 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab68c11-d336-485a-8477-d00a8caa4e5a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd05409b38ae346458abfe41ebda1c59e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:36.346268 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab68c11-d336-485a-8477-d00a8caa4e5a service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:36.346659 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab68c11-d336-485a-8477-d00a8caa4e5a service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:36.347025 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab68c11-d336-485a-8477-d00a8caa4e5a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:36.406598 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab68c11-d336-485a-8477-d00a8caa4e5a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:36.408107 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab68c11-d336-485a-8477-d00a8caa4e5a service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:36.417044 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1319/2640] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:36:36 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:36:37.924615 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0394176-682e-4d7a-a1be-5a5fb987396f service cinder] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:37.933164 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0394176-682e-4d7a-a1be-5a5fb987396f service cinder] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:37.936882 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c502fd67-978f-417a-a181-8f87490469c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd05409b38ae346458abfe41ebda1c59e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:37.937479 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c502fd67-978f-417a-a181-8f87490469c9 service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:37.944419 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c502fd67-978f-417a-a181-8f87490469c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:37.944419 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c502fd67-978f-417a-a181-8f87490469c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:37.947126 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c502fd67-978f-417a-a181-8f87490469c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:37.948585 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c502fd67-978f-417a-a181-8f87490469c9 service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:37.958078 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1322/2641] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:36:37 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:36:40.574140 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab68c11-d336-485a-8477-d00a8caa4e5a service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:40.575609 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-6ab68c11-d336-485a-8477-d00a8caa4e5a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:36:40.631831 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab68c11-d336-485a-8477-d00a8caa4e5a service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:40.635623 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2824e3b-f4b9-411c-8d21-6d2db11c0012 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:40.636287 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2824e3b-f4b9-411c-8d21-6d2db11c0012 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:40.636545 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2824e3b-f4b9-411c-8d21-6d2db11c0012 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:40.637478 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2824e3b-f4b9-411c-8d21-6d2db11c0012 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:40.638701 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-f2824e3b-f4b9-411c-8d21-6d2db11c0012 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:36:40.700775 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2824e3b-f4b9-411c-8d21-6d2db11c0012 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:40.702295 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2824e3b-f4b9-411c-8d21-6d2db11c0012 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:40.712138 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1320/2642] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:36:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:36:42.674142 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c502fd67-978f-417a-a181-8f87490469c9 service cinder] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:42.685936 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c502fd67-978f-417a-a181-8f87490469c9 service cinder] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:42.691562 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a768b9d-b66a-4ead-8770-b57017432b63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:42.692496 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a768b9d-b66a-4ead-8770-b57017432b63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:42.692996 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a768b9d-b66a-4ead-8770-b57017432b63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:42.693334 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a768b9d-b66a-4ead-8770-b57017432b63 admin admin] PATH_INFO: `/v3/users/ba8512a78bd04527a91fc6cdada51aa3` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:42.694580 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a768b9d-b66a-4ead-8770-b57017432b63 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba8512a78bd04527a91fc6cdada51aa3)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:42.696667 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a768b9d-b66a-4ead-8770-b57017432b63 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:42.876212 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-5a768b9d-b66a-4ead-8770-b57017432b63 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba8512a78bd04527a91fc6cdada51aa3'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:42.883259 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-5a768b9d-b66a-4ead-8770-b57017432b63 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba8512a78bd04527a91fc6cdada51aa3'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:42.896714 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-5a768b9d-b66a-4ead-8770-b57017432b63 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba8512a78bd04527a91fc6cdada51aa3'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:42.898725 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-5a768b9d-b66a-4ead-8770-b57017432b63 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba8512a78bd04527a91fc6cdada51aa3'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:42.899893 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-5a768b9d-b66a-4ead-8770-b57017432b63 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba8512a78bd04527a91fc6cdada51aa3'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:43.006231 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1323/2643] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 17 19:36:42 2022] DELETE /identity/v3/users/ba8512a78bd04527a91fc6cdada51aa3 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 17 19:36:43.010734 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89a4624-ce02-4dc5-bde1-39de4961f920 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:43.011207 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89a4624-ce02-4dc5-bde1-39de4961f920 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:43.011302 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89a4624-ce02-4dc5-bde1-39de4961f920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:43.027753 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-e89a4624-ce02-4dc5-bde1-39de4961f920 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:36:43.028936 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-e89a4624-ce02-4dc5-bde1-39de4961f920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:36:43.088826 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1321/2644] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:36:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:36:43.099083 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a768b9d-b66a-4ead-8770-b57017432b63 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:43.100668 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-5a768b9d-b66a-4ead-8770-b57017432b63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:36:43.157565 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a768b9d-b66a-4ead-8770-b57017432b63 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:43.161255 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6a555e-736c-4faf-b5c4-bd6106039924 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:43.161894 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6a555e-736c-4faf-b5c4-bd6106039924 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:43.162173 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6a555e-736c-4faf-b5c4-bd6106039924 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:43.162444 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6a555e-736c-4faf-b5c4-bd6106039924 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:43.171753 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6a555e-736c-4faf-b5c4-bd6106039924 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:43.173178 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6a555e-736c-4faf-b5c4-bd6106039924 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:43.182553 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1324/2645] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:36:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:36:43.705278 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89a4624-ce02-4dc5-bde1-39de4961f920 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:43.706798 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-e89a4624-ce02-4dc5-bde1-39de4961f920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:36:43.718868 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89a4624-ce02-4dc5-bde1-39de4961f920 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:43.723729 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545b4d68-7af7-4393-979e-169abfd26633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:43.725034 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545b4d68-7af7-4393-979e-169abfd26633 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:43.725518 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545b4d68-7af7-4393-979e-169abfd26633 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:43.725888 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545b4d68-7af7-4393-979e-169abfd26633 admin admin] PATH_INFO: `/v3/projects/f7b05cc7a00a4971945f7e89ce0a8878` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:43.727307 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545b4d68-7af7-4393-979e-169abfd26633 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7b05cc7a00a4971945f7e89ce0a8878)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:43.729685 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545b4d68-7af7-4393-979e-169abfd26633 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:43.845805 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-545b4d68-7af7-4393-979e-169abfd26633 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7b05cc7a00a4971945f7e89ce0a8878', 'request_id': 'req-545b4d68-7af7-4393-979e-169abfd26633'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:43.851206 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-545b4d68-7af7-4393-979e-169abfd26633 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7b05cc7a00a4971945f7e89ce0a8878', 'request_id': 'req-545b4d68-7af7-4393-979e-169abfd26633'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:43.919052 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-545b4d68-7af7-4393-979e-169abfd26633 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7b05cc7a00a4971945f7e89ce0a8878', 'request_id': 'req-545b4d68-7af7-4393-979e-169abfd26633'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:43.933726 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-545b4d68-7af7-4393-979e-169abfd26633 admin admin] The token cache is being invalidate because project f7b05cc7a00a4971945f7e89ce0a8878 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 17 19:36:43.934355 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-545b4d68-7af7-4393-979e-169abfd26633 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:43.935020 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-545b4d68-7af7-4393-979e-169abfd26633 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:43.937254 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1322/2646] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 17 19:36:43 2022] DELETE /identity/v3/projects/f7b05cc7a00a4971945f7e89ce0a8878 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:36:43.941225 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6a555e-736c-4faf-b5c4-bd6106039924 service neutron] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:43.959842 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6a555e-736c-4faf-b5c4-bd6106039924 service neutron] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:44.040756 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2c3ace-9b23-4089-a849-1867efd31f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:44.041920 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2c3ace-9b23-4089-a849-1867efd31f60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:44.042372 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2c3ace-9b23-4089-a849-1867efd31f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:44.042847 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2c3ace-9b23-4089-a849-1867efd31f60 admin admin] PATH_INFO: `/v3/users/7c086461ecf14627bcaf0a7a9498fa93` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:44.044235 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2c3ace-9b23-4089-a849-1867efd31f60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c086461ecf14627bcaf0a7a9498fa93)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:44.047049 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2c3ace-9b23-4089-a849-1867efd31f60 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:44.265683 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-6b2c3ace-9b23-4089-a849-1867efd31f60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c086461ecf14627bcaf0a7a9498fa93'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:44.272385 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-6b2c3ace-9b23-4089-a849-1867efd31f60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c086461ecf14627bcaf0a7a9498fa93'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:44.285093 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-6b2c3ace-9b23-4089-a849-1867efd31f60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c086461ecf14627bcaf0a7a9498fa93'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:44.286561 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-6b2c3ace-9b23-4089-a849-1867efd31f60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c086461ecf14627bcaf0a7a9498fa93'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:44.286561 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-6b2c3ace-9b23-4089-a849-1867efd31f60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c086461ecf14627bcaf0a7a9498fa93'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:44.310012 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545b4d68-7af7-4393-979e-169abfd26633 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:44.311707 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-545b4d68-7af7-4393-979e-169abfd26633 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:36:44.325100 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545b4d68-7af7-4393-979e-169abfd26633 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:44.329900 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2309976-7497-4e65-be36-358c8524d1dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:44.330762 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2309976-7497-4e65-be36-358c8524d1dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:44.331040 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2309976-7497-4e65-be36-358c8524d1dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:44.331338 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2309976-7497-4e65-be36-358c8524d1dc admin admin] PATH_INFO: `/v3/users/6545bb5d5eb24c199b5d89091b2c1545` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:44.332557 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2309976-7497-4e65-be36-358c8524d1dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=6545bb5d5eb24c199b5d89091b2c1545)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:44.334477 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2309976-7497-4e65-be36-358c8524d1dc admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:44.349000 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1325/2647] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 17 19:36:43 2022] DELETE /identity/v3/users/7c086461ecf14627bcaf0a7a9498fa93 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:36:44.582999 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c2309976-7497-4e65-be36-358c8524d1dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6545bb5d5eb24c199b5d89091b2c1545'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:44.584271 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c2309976-7497-4e65-be36-358c8524d1dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6545bb5d5eb24c199b5d89091b2c1545'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:44.604514 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c2309976-7497-4e65-be36-358c8524d1dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6545bb5d5eb24c199b5d89091b2c1545'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:44.686353 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c2309976-7497-4e65-be36-358c8524d1dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6545bb5d5eb24c199b5d89091b2c1545'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:44.697395 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c2309976-7497-4e65-be36-358c8524d1dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6545bb5d5eb24c199b5d89091b2c1545'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:44.704207 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1323/2648] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 17 19:36:44 2022] DELETE /identity/v3/users/6545bb5d5eb24c199b5d89091b2c1545 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 17 19:36:44.847026 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2c3ace-9b23-4089-a849-1867efd31f60 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:44.848951 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-6b2c3ace-9b23-4089-a849-1867efd31f60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:36:44.920911 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2c3ace-9b23-4089-a849-1867efd31f60 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:44.958261 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2309976-7497-4e65-be36-358c8524d1dc admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:44.959679 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c2309976-7497-4e65-be36-358c8524d1dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:36:44.982442 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e24f71c-69d2-4557-a0cf-45d9c2f9582f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:44.983315 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e24f71c-69d2-4557-a0cf-45d9c2f9582f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:44.983642 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e24f71c-69d2-4557-a0cf-45d9c2f9582f admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:44.983921 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e24f71c-69d2-4557-a0cf-45d9c2f9582f admin admin] PATH_INFO: `/v3/projects/be9bfffcacb04ff3a2c1e8677166ca30` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:44.985272 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e24f71c-69d2-4557-a0cf-45d9c2f9582f admin admin] RBAC: Authorizing `identity:delete_project(project_id=be9bfffcacb04ff3a2c1e8677166ca30)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:44.987081 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e24f71c-69d2-4557-a0cf-45d9c2f9582f admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:45.025131 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2309976-7497-4e65-be36-358c8524d1dc admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:45.030381 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed48fa4-ba76-4abd-a03a-4748ba153a58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:45.031265 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed48fa4-ba76-4abd-a03a-4748ba153a58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:45.031531 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed48fa4-ba76-4abd-a03a-4748ba153a58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:45.031806 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed48fa4-ba76-4abd-a03a-4748ba153a58 admin admin] PATH_INFO: `/v3/projects/b23e9f1907a6406785be000e1373dc41` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:45.032987 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed48fa4-ba76-4abd-a03a-4748ba153a58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b23e9f1907a6406785be000e1373dc41)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:45.034896 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed48fa4-ba76-4abd-a03a-4748ba153a58 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:45.104676 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-9e24f71c-69d2-4557-a0cf-45d9c2f9582f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be9bfffcacb04ff3a2c1e8677166ca30', 'request_id': 'req-9e24f71c-69d2-4557-a0cf-45d9c2f9582f'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:45.110958 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-9e24f71c-69d2-4557-a0cf-45d9c2f9582f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be9bfffcacb04ff3a2c1e8677166ca30', 'request_id': 'req-9e24f71c-69d2-4557-a0cf-45d9c2f9582f'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:45.116626 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-9e24f71c-69d2-4557-a0cf-45d9c2f9582f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be9bfffcacb04ff3a2c1e8677166ca30', 'request_id': 'req-9e24f71c-69d2-4557-a0cf-45d9c2f9582f'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:45.132098 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1ed48fa4-ba76-4abd-a03a-4748ba153a58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b23e9f1907a6406785be000e1373dc41', 'request_id': 'req-1ed48fa4-ba76-4abd-a03a-4748ba153a58'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:45.137373 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1ed48fa4-ba76-4abd-a03a-4748ba153a58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b23e9f1907a6406785be000e1373dc41', 'request_id': 'req-1ed48fa4-ba76-4abd-a03a-4748ba153a58'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:45.188659 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-9e24f71c-69d2-4557-a0cf-45d9c2f9582f admin admin] The token cache is being invalidate because project be9bfffcacb04ff3a2c1e8677166ca30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 17 19:36:45.188947 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-9e24f71c-69d2-4557-a0cf-45d9c2f9582f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:45.189553 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-9e24f71c-69d2-4557-a0cf-45d9c2f9582f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:45.191117 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1326/2649] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 17 19:36:44 2022] DELETE /identity/v3/projects/be9bfffcacb04ff3a2c1e8677166ca30 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:36:45.194920 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e24f71c-69d2-4557-a0cf-45d9c2f9582f admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:45.196224 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-9e24f71c-69d2-4557-a0cf-45d9c2f9582f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:36:45.234548 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1ed48fa4-ba76-4abd-a03a-4748ba153a58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b23e9f1907a6406785be000e1373dc41', 'request_id': 'req-1ed48fa4-ba76-4abd-a03a-4748ba153a58'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:45.246707 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1ed48fa4-ba76-4abd-a03a-4748ba153a58 admin admin] The token cache is being invalidate because project b23e9f1907a6406785be000e1373dc41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 17 19:36:45.246990 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1ed48fa4-ba76-4abd-a03a-4748ba153a58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:45.247660 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1ed48fa4-ba76-4abd-a03a-4748ba153a58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:45.250206 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1324/2650] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 17 19:36:44 2022] DELETE /identity/v3/projects/b23e9f1907a6406785be000e1373dc41 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:36:45.255581 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e70db9e-2977-4b3c-a3c1-69733e81403d None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:45.255807 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e70db9e-2977-4b3c-a3c1-69733e81403d None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:45.256083 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e70db9e-2977-4b3c-a3c1-69733e81403d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:45.257611 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e24f71c-69d2-4557-a0cf-45d9c2f9582f admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:45.274779 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-3e70db9e-2977-4b3c-a3c1-69733e81403d None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:36:45.275993 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-3e70db9e-2977-4b3c-a3c1-69733e81403d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:36:45.312777 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0008e0-b87f-4cf9-a0e2-f41a6ebf16db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:45.313919 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0008e0-b87f-4cf9-a0e2-f41a6ebf16db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:45.314197 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0008e0-b87f-4cf9-a0e2-f41a6ebf16db admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:45.314470 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0008e0-b87f-4cf9-a0e2-f41a6ebf16db admin admin] PATH_INFO: `/v3/users/8a329069307e451aafd8caee5741214a` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:45.315692 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0008e0-b87f-4cf9-a0e2-f41a6ebf16db admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a329069307e451aafd8caee5741214a)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:45.318072 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0008e0-b87f-4cf9-a0e2-f41a6ebf16db admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:45.335422 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1325/2651] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:36:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:36:45.343202 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bb3e37-1d51-4417-a28f-86a481c3ffdf None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:45.343517 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bb3e37-1d51-4417-a28f-86a481c3ffdf None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:45.343780 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bb3e37-1d51-4417-a28f-86a481c3ffdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:45.362798 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-b9bb3e37-1d51-4417-a28f-86a481c3ffdf None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:36:45.364016 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b9bb3e37-1d51-4417-a28f-86a481c3ffdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:36:45.371873 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1326/2652] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:36:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:36:45.376306 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bb3e37-1d51-4417-a28f-86a481c3ffdf None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:45.384890 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bb3e37-1d51-4417-a28f-86a481c3ffdf None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:45.389953 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a228d2db-9f94-4847-8672-15ccddc0cc03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:45.390563 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a228d2db-9f94-4847-8672-15ccddc0cc03 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:45.390835 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a228d2db-9f94-4847-8672-15ccddc0cc03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:45.391108 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a228d2db-9f94-4847-8672-15ccddc0cc03 admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:45.391830 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a228d2db-9f94-4847-8672-15ccddc0cc03 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:36:45.392127 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a228d2db-9f94-4847-8672-15ccddc0cc03 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:45.393848 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a228d2db-9f94-4847-8672-15ccddc0cc03 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:45.405185 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1327/2653] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 17 19:36:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:36:45.409299 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a228d2db-9f94-4847-8672-15ccddc0cc03 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:45.418612 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a228d2db-9f94-4847-8672-15ccddc0cc03 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:45.423786 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90da2b38-eac9-4e58-ac1a-f8dca91f2909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:45.424403 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da2b38-eac9-4e58-ac1a-f8dca91f2909 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:45.424678 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da2b38-eac9-4e58-ac1a-f8dca91f2909 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:45.424947 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da2b38-eac9-4e58-ac1a-f8dca91f2909 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:45.425603 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90da2b38-eac9-4e58-ac1a-f8dca91f2909 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:45.427642 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90da2b38-eac9-4e58-ac1a-f8dca91f2909 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:45.475441 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-be0008e0-b87f-4cf9-a0e2-f41a6ebf16db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a329069307e451aafd8caee5741214a'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:45.482194 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-be0008e0-b87f-4cf9-a0e2-f41a6ebf16db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a329069307e451aafd8caee5741214a'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:45.493836 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-be0008e0-b87f-4cf9-a0e2-f41a6ebf16db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a329069307e451aafd8caee5741214a'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:45.494507 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-be0008e0-b87f-4cf9-a0e2-f41a6ebf16db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a329069307e451aafd8caee5741214a'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:45.495131 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-be0008e0-b87f-4cf9-a0e2-f41a6ebf16db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a329069307e451aafd8caee5741214a'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:45.515600 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1328/2654] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 17 19:36:45 2022] POST /identity/v3/projects => generated 377 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 17 19:36:45.519640 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90da2b38-eac9-4e58-ac1a-f8dca91f2909 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:45.521203 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-90da2b38-eac9-4e58-ac1a-f8dca91f2909 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:36:45.577482 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1327/2655] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 17 19:36:45 2022] DELETE /identity/v3/users/8a329069307e451aafd8caee5741214a => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 17 19:36:45.581813 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90da2b38-eac9-4e58-ac1a-f8dca91f2909 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:45.637504 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cb2369-c599-4a3c-8802-08fcdabb4b89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:45.638205 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cb2369-c599-4a3c-8802-08fcdabb4b89 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:45.638483 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cb2369-c599-4a3c-8802-08fcdabb4b89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:45.638755 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cb2369-c599-4a3c-8802-08fcdabb4b89 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:45.639457 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cb2369-c599-4a3c-8802-08fcdabb4b89 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:45.641608 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cb2369-c599-4a3c-8802-08fcdabb4b89 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:45.729258 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1329/2656] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 17 19:36:45 2022] POST /identity/v3/users => generated 289 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 17 19:36:45.730781 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0008e0-b87f-4cf9-a0e2-f41a6ebf16db admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:45.741281 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0008e0-b87f-4cf9-a0e2-f41a6ebf16db admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:45.747197 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf97386-504f-4361-88b6-ce60c405b871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:45.747624 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf97386-504f-4361-88b6-ce60c405b871 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:45.747903 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf97386-504f-4361-88b6-ce60c405b871 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:45.748178 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf97386-504f-4361-88b6-ce60c405b871 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:45.749484 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf97386-504f-4361-88b6-ce60c405b871 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:45.751079 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf97386-504f-4361-88b6-ce60c405b871 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:45.761016 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1328/2657] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:36:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:36:45.765101 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cb2369-c599-4a3c-8802-08fcdabb4b89 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:45.775028 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cb2369-c599-4a3c-8802-08fcdabb4b89 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:45.780262 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17f7a5f-f78d-4069-ad88-036b4f416736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:45.780909 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17f7a5f-f78d-4069-ad88-036b4f416736 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:45.781184 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17f7a5f-f78d-4069-ad88-036b4f416736 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:45.781454 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17f7a5f-f78d-4069-ad88-036b4f416736 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:45.782165 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17f7a5f-f78d-4069-ad88-036b4f416736 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:45.783862 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17f7a5f-f78d-4069-ad88-036b4f416736 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:45.793827 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1330/2658] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:36:45 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:36:45.798327 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf97386-504f-4361-88b6-ce60c405b871 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:45.808183 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf97386-504f-4361-88b6-ce60c405b871 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:45.812900 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51516090-62db-4a32-9ba8-5ff05a7c87d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:45.823863 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51516090-62db-4a32-9ba8-5ff05a7c87d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:45.823863 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51516090-62db-4a32-9ba8-5ff05a7c87d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:45.823863 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51516090-62db-4a32-9ba8-5ff05a7c87d2 admin admin] PATH_INFO: `/v3/projects/2c9294dbfdbb463baf41d1fea4ae1fbc/users/d823563863d8495f8e0680da4e879439/roles/114e6a16e63447d1924707ace5aace25` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:45.828748 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51516090-62db-4a32-9ba8-5ff05a7c87d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c9294dbfdbb463baf41d1fea4ae1fbc, user_id=d823563863d8495f8e0680da4e879439, role_id=114e6a16e63447d1924707ace5aace25)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:45.831383 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51516090-62db-4a32-9ba8-5ff05a7c87d2 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:45.903577 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1329/2659] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:36:45 2022] PUT /identity/v3/projects/2c9294dbfdbb463baf41d1fea4ae1fbc/users/d823563863d8495f8e0680da4e879439/roles/114e6a16e63447d1924707ace5aace25 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:36:45.913742 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76caa3f-4295-4935-9a56-8edf2c28e1b7 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:45.914024 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76caa3f-4295-4935-9a56-8edf2c28e1b7 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:45.914300 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76caa3f-4295-4935-9a56-8edf2c28e1b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:45.932136 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-d76caa3f-4295-4935-9a56-8edf2c28e1b7 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:36:45.933322 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d76caa3f-4295-4935-9a56-8edf2c28e1b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:36:45.992767 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1331/2660] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:36:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:36:46.004420 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51516090-62db-4a32-9ba8-5ff05a7c87d2 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:46.010637 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-51516090-62db-4a32-9ba8-5ff05a7c87d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:36:46.062899 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51516090-62db-4a32-9ba8-5ff05a7c87d2 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:46.066497 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549b66db-af98-45dd-a7b6-51c9e7e26020 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:46.067185 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549b66db-af98-45dd-a7b6-51c9e7e26020 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:46.067507 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549b66db-af98-45dd-a7b6-51c9e7e26020 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:46.067795 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549b66db-af98-45dd-a7b6-51c9e7e26020 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:46.077027 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549b66db-af98-45dd-a7b6-51c9e7e26020 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:46.078492 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549b66db-af98-45dd-a7b6-51c9e7e26020 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:46.082199 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76caa3f-4295-4935-9a56-8edf2c28e1b7 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:46.086781 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d76caa3f-4295-4935-9a56-8edf2c28e1b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:36:46.090042 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1330/2661] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:36:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:36:46.100287 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76caa3f-4295-4935-9a56-8edf2c28e1b7 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:46.105821 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e3979e-f023-4c3c-a98f-fd56de23b6b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:46.106720 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e3979e-f023-4c3c-a98f-fd56de23b6b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:46.107025 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e3979e-f023-4c3c-a98f-fd56de23b6b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:46.107302 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e3979e-f023-4c3c-a98f-fd56de23b6b9 admin admin] PATH_INFO: `/v3/projects/d655823fda984fc78354814f5e7f7580` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:46.109298 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e3979e-f023-4c3c-a98f-fd56de23b6b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d655823fda984fc78354814f5e7f7580)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:46.111520 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e3979e-f023-4c3c-a98f-fd56de23b6b9 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:46.224268 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-80e3979e-f023-4c3c-a98f-fd56de23b6b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd655823fda984fc78354814f5e7f7580', 'request_id': 'req-80e3979e-f023-4c3c-a98f-fd56de23b6b9'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:46.229877 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-80e3979e-f023-4c3c-a98f-fd56de23b6b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd655823fda984fc78354814f5e7f7580', 'request_id': 'req-80e3979e-f023-4c3c-a98f-fd56de23b6b9'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:46.303218 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-80e3979e-f023-4c3c-a98f-fd56de23b6b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd655823fda984fc78354814f5e7f7580', 'request_id': 'req-80e3979e-f023-4c3c-a98f-fd56de23b6b9'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:46.314417 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-80e3979e-f023-4c3c-a98f-fd56de23b6b9 admin admin] The token cache is being invalidate because project d655823fda984fc78354814f5e7f7580 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 17 19:36:46.314700 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-80e3979e-f023-4c3c-a98f-fd56de23b6b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:46.315338 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-80e3979e-f023-4c3c-a98f-fd56de23b6b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:36:46.316932 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1332/2662] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 17 19:36:46 2022] DELETE /identity/v3/projects/d655823fda984fc78354814f5e7f7580 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:36:46.321857 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54de6e5d-b5ea-453b-b8c3-0556cc5ecc8f None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:46.322143 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54de6e5d-b5ea-453b-b8c3-0556cc5ecc8f None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:46.322416 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54de6e5d-b5ea-453b-b8c3-0556cc5ecc8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:46.345161 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-54de6e5d-b5ea-453b-b8c3-0556cc5ecc8f None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:36:46.346385 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-54de6e5d-b5ea-453b-b8c3-0556cc5ecc8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:36:46.406648 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1331/2663] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:36:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 17 19:36:46.414371 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4002d69-259f-475b-9d5b-ceb1a4a3a59a None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:46.414827 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4002d69-259f-475b-9d5b-ceb1a4a3a59a None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:46.414933 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4002d69-259f-475b-9d5b-ceb1a4a3a59a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:46.433019 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-b4002d69-259f-475b-9d5b-ceb1a4a3a59a None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:36:46.434124 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b4002d69-259f-475b-9d5b-ceb1a4a3a59a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:36:46.442519 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1333/2664] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:36:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:36:46.446751 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54de6e5d-b5ea-453b-b8c3-0556cc5ecc8f None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:46.456926 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54de6e5d-b5ea-453b-b8c3-0556cc5ecc8f None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:46.461637 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1515bef1-9f90-4ade-863b-947ef001cd3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:46.462282 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1515bef1-9f90-4ade-863b-947ef001cd3c admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:46.462563 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1515bef1-9f90-4ade-863b-947ef001cd3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:46.462835 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1515bef1-9f90-4ade-863b-947ef001cd3c admin admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:46.463593 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1515bef1-9f90-4ade-863b-947ef001cd3c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:36:46.463897 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1515bef1-9f90-4ade-863b-947ef001cd3c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:46.465481 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1515bef1-9f90-4ade-863b-947ef001cd3c admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:46.478287 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1332/2665] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 17 19:36:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:36:46.483267 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4002d69-259f-475b-9d5b-ceb1a4a3a59a None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:46.492406 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4002d69-259f-475b-9d5b-ceb1a4a3a59a None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:46.497539 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c97713-17a1-4306-873a-c7ba689c734f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:46.498271 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c97713-17a1-4306-873a-c7ba689c734f admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:46.499143 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c97713-17a1-4306-873a-c7ba689c734f admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:46.499420 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c97713-17a1-4306-873a-c7ba689c734f admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:46.500264 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c97713-17a1-4306-873a-c7ba689c734f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:46.502169 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c97713-17a1-4306-873a-c7ba689c734f admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:46.633644 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1334/2666] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 17 19:36:46 2022] POST /identity/v3/projects => generated 385 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 17 19:36:46.634813 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1515bef1-9f90-4ade-863b-947ef001cd3c admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:46.645285 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1515bef1-9f90-4ade-863b-947ef001cd3c admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:46.700166 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e94ecae-b5f0-47e0-9818-14ff0dc9ecd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:46.700836 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e94ecae-b5f0-47e0-9818-14ff0dc9ecd9 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:46.701138 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e94ecae-b5f0-47e0-9818-14ff0dc9ecd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:46.701487 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e94ecae-b5f0-47e0-9818-14ff0dc9ecd9 admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:46.702317 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e94ecae-b5f0-47e0-9818-14ff0dc9ecd9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:46.704401 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e94ecae-b5f0-47e0-9818-14ff0dc9ecd9 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:46.783580 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1333/2667] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 17 19:36:46 2022] POST /identity/v3/users => generated 293 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:36:46.787627 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c97713-17a1-4306-873a-c7ba689c734f admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:46.799182 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c97713-17a1-4306-873a-c7ba689c734f admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:46.803974 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7205269-2952-4990-b809-3552717621cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:46.804613 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7205269-2952-4990-b809-3552717621cb admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:46.805193 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7205269-2952-4990-b809-3552717621cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:46.805193 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7205269-2952-4990-b809-3552717621cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:46.805898 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7205269-2952-4990-b809-3552717621cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:46.807674 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7205269-2952-4990-b809-3552717621cb admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:46.818542 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1335/2668] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:36:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:36:46.822551 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e94ecae-b5f0-47e0-9818-14ff0dc9ecd9 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:46.832695 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e94ecae-b5f0-47e0-9818-14ff0dc9ecd9 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:46.837482 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5a9103-d3f9-40fa-9711-5d82ce2c6c14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:46.838143 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5a9103-d3f9-40fa-9711-5d82ce2c6c14 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:46.838423 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5a9103-d3f9-40fa-9711-5d82ce2c6c14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:46.838746 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5a9103-d3f9-40fa-9711-5d82ce2c6c14 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:46.839444 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5a9103-d3f9-40fa-9711-5d82ce2c6c14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:46.841073 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5a9103-d3f9-40fa-9711-5d82ce2c6c14 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:46.851580 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1334/2669] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:36:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:36:46.859215 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7205269-2952-4990-b809-3552717621cb admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:46.871288 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7205269-2952-4990-b809-3552717621cb admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:46.876215 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baf2359-ccd7-41f6-a236-261e90427784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:46.877116 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baf2359-ccd7-41f6-a236-261e90427784 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:46.877390 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baf2359-ccd7-41f6-a236-261e90427784 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:46.877660 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baf2359-ccd7-41f6-a236-261e90427784 admin admin] PATH_INFO: `/v3/projects/de75551df7bb4cadbaf622b4966540fb/users/176f09a4dd2349a8aaa45224e8b88023/roles/114e6a16e63447d1924707ace5aace25` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:46.892030 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baf2359-ccd7-41f6-a236-261e90427784 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de75551df7bb4cadbaf622b4966540fb, user_id=176f09a4dd2349a8aaa45224e8b88023, role_id=114e6a16e63447d1924707ace5aace25)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:46.894695 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baf2359-ccd7-41f6-a236-261e90427784 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:46.977620 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1336/2670] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:36:46 2022] PUT /identity/v3/projects/de75551df7bb4cadbaf622b4966540fb/users/176f09a4dd2349a8aaa45224e8b88023/roles/114e6a16e63447d1924707ace5aace25 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:36:46.984591 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f987ec-c909-4147-9ee0-d8fd15d3ca30 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:46.984883 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f987ec-c909-4147-9ee0-d8fd15d3ca30 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:46.985160 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f987ec-c909-4147-9ee0-d8fd15d3ca30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:47.002808 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-82f987ec-c909-4147-9ee0-d8fd15d3ca30 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:36:47.004132 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-82f987ec-c909-4147-9ee0-d8fd15d3ca30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:36:47.061673 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1335/2671] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:36:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:36:47.071198 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baf2359-ccd7-41f6-a236-261e90427784 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:47.072879 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-0baf2359-ccd7-41f6-a236-261e90427784 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:36:47.136994 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baf2359-ccd7-41f6-a236-261e90427784 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:47.140694 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7d8179-9567-4e0a-9204-ec509486228d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:47.141356 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7d8179-9567-4e0a-9204-ec509486228d service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:47.141678 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7d8179-9567-4e0a-9204-ec509486228d service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:47.141945 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7d8179-9567-4e0a-9204-ec509486228d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:47.151054 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7d8179-9567-4e0a-9204-ec509486228d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:47.152533 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7d8179-9567-4e0a-9204-ec509486228d service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:47.162249 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1337/2672] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:36:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:36:51.934202 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8e89c4-e972-4f58-bd67-f39928bd46ed None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:51.934705 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8e89c4-e972-4f58-bd67-f39928bd46ed None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:51.934898 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8e89c4-e972-4f58-bd67-f39928bd46ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:51.980580 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-3d8e89c4-e972-4f58-bd67-f39928bd46ed None None] MFA Rules not processed for user `d823563863d8495f8e0680da4e879439`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:36:51.982069 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-3d8e89c4-e972-4f58-bd67-f39928bd46ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:36:52.127161 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7d8179-9567-4e0a-9204-ec509486228d service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:52.131927 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-fa7d8179-9567-4e0a-9204-ec509486228d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:36:52.134203 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1336/2673] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:36:51 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:36:52.241921 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7d8179-9567-4e0a-9204-ec509486228d service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:52.248192 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281fa0c7-1619-472d-b472-ff1db5118a1b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c1f878b0dc4d63a4f5dcb12fac1dc1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:52.249130 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281fa0c7-1619-472d-b472-ff1db5118a1b service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:52.249577 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281fa0c7-1619-472d-b472-ff1db5118a1b service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:52.250067 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281fa0c7-1619-472d-b472-ff1db5118a1b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:52.259630 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281fa0c7-1619-472d-b472-ff1db5118a1b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:52.262427 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281fa0c7-1619-472d-b472-ff1db5118a1b service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:52.273096 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1338/2674] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:36:52 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 17 19:36:53.298979 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811e93e3-28d4-4c62-993a-454d3aa417e9 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:53.299899 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811e93e3-28d4-4c62-993a-454d3aa417e9 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:53.300350 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811e93e3-28d4-4c62-993a-454d3aa417e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:53.334388 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-811e93e3-28d4-4c62-993a-454d3aa417e9 None None] MFA Rules not processed for user `176f09a4dd2349a8aaa45224e8b88023`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:36:53.335959 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-811e93e3-28d4-4c62-993a-454d3aa417e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:36:53.419153 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1337/2675] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:36:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:36:53.423801 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281fa0c7-1619-472d-b472-ff1db5118a1b service nova] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:53.435047 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281fa0c7-1619-472d-b472-ff1db5118a1b service nova] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:53.440715 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae30124-3994-4b60-b116-c0527cff56b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:53.441393 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae30124-3994-4b60-b116-c0527cff56b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:53.441678 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae30124-3994-4b60-b116-c0527cff56b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:53.441954 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae30124-3994-4b60-b116-c0527cff56b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:53.442663 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae30124-3994-4b60-b116-c0527cff56b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:53.444638 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae30124-3994-4b60-b116-c0527cff56b3 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:53.537766 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1339/2676] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 17 19:36:53 2022] POST /identity/v3/projects => generated 385 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:36:53.541782 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811e93e3-28d4-4c62-993a-454d3aa417e9 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:53.554218 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811e93e3-28d4-4c62-993a-454d3aa417e9 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:53.626295 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aac95d8-28ef-4177-a354-8dc00fa3409a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:53.626990 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aac95d8-28ef-4177-a354-8dc00fa3409a admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:53.627276 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aac95d8-28ef-4177-a354-8dc00fa3409a admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:53.627832 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aac95d8-28ef-4177-a354-8dc00fa3409a admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:53.633075 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aac95d8-28ef-4177-a354-8dc00fa3409a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:53.633075 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aac95d8-28ef-4177-a354-8dc00fa3409a admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:53.715657 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1338/2677] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 17 19:36:53 2022] POST /identity/v3/users => generated 299 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:36:53.719669 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae30124-3994-4b60-b116-c0527cff56b3 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:53.731581 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae30124-3994-4b60-b116-c0527cff56b3 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:53.737275 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b5d738-cae8-433f-b4b2-82b4bfc27595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:53.742207 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b5d738-cae8-433f-b4b2-82b4bfc27595 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:53.742207 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b5d738-cae8-433f-b4b2-82b4bfc27595 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:53.742207 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b5d738-cae8-433f-b4b2-82b4bfc27595 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:53.742207 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b5d738-cae8-433f-b4b2-82b4bfc27595 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:53.742207 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b5d738-cae8-433f-b4b2-82b4bfc27595 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:53.756659 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1340/2678] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:36:53 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 17 19:36:53.760172 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aac95d8-28ef-4177-a354-8dc00fa3409a admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:53.771105 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aac95d8-28ef-4177-a354-8dc00fa3409a admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:53.776676 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bd0090-5a96-4646-8836-9091807658fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:53.777536 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bd0090-5a96-4646-8836-9091807658fa admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:53.777822 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bd0090-5a96-4646-8836-9091807658fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:53.778102 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bd0090-5a96-4646-8836-9091807658fa admin admin] PATH_INFO: `/v3/projects/61f95072dbf14035b151663d3ec2d626/users/e080beaac56d42e5ae0f1a4fdea170f3/roles/60af5088793e4379b88821a86552a344` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:53.802147 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bd0090-5a96-4646-8836-9091807658fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=61f95072dbf14035b151663d3ec2d626, user_id=e080beaac56d42e5ae0f1a4fdea170f3, role_id=60af5088793e4379b88821a86552a344)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:53.805090 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bd0090-5a96-4646-8836-9091807658fa admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:53.882448 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1339/2679] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:36:53 2022] PUT /identity/v3/projects/61f95072dbf14035b151663d3ec2d626/users/e080beaac56d42e5ae0f1a4fdea170f3/roles/60af5088793e4379b88821a86552a344 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 17 19:36:53.886456 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b5d738-cae8-433f-b4b2-82b4bfc27595 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:53.896604 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b5d738-cae8-433f-b4b2-82b4bfc27595 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:53.960611 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba087b1-9b97-4b83-9b15-55709048d3a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:53.961371 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba087b1-9b97-4b83-9b15-55709048d3a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:53.961715 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba087b1-9b97-4b83-9b15-55709048d3a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:53.962055 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba087b1-9b97-4b83-9b15-55709048d3a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:53.962834 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba087b1-9b97-4b83-9b15-55709048d3a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:53.964729 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba087b1-9b97-4b83-9b15-55709048d3a0 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:53.975442 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1341/2680] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:36:53 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:36:53.979871 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bd0090-5a96-4646-8836-9091807658fa admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:53.990558 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bd0090-5a96-4646-8836-9091807658fa admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:53.995319 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e4e23e-06e5-44e6-8349-e15fd59087d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:53.996385 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e4e23e-06e5-44e6-8349-e15fd59087d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:53.996785 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e4e23e-06e5-44e6-8349-e15fd59087d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:53.997069 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e4e23e-06e5-44e6-8349-e15fd59087d4 admin admin] PATH_INFO: `/v3/projects/61f95072dbf14035b151663d3ec2d626/users/e080beaac56d42e5ae0f1a4fdea170f3/roles/60af5088793e4379b88821a86552a344` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:53.999931 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e4e23e-06e5-44e6-8349-e15fd59087d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=61f95072dbf14035b151663d3ec2d626, user_id=e080beaac56d42e5ae0f1a4fdea170f3, role_id=60af5088793e4379b88821a86552a344)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:54.002605 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e4e23e-06e5-44e6-8349-e15fd59087d4 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:54.013616 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1340/2681] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:36:53 2022] PUT /identity/v3/projects/61f95072dbf14035b151663d3ec2d626/users/e080beaac56d42e5ae0f1a4fdea170f3/roles/60af5088793e4379b88821a86552a344 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:36:55.007101 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f86df2f-5303-433e-be50-02121401ffee None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:55.007642 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f86df2f-5303-433e-be50-02121401ffee None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:55.007746 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f86df2f-5303-433e-be50-02121401ffee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:55.026075 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-8f86df2f-5303-433e-be50-02121401ffee None None] MFA Rules not processed for user `d823563863d8495f8e0680da4e879439`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:36:55.027341 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-8f86df2f-5303-433e-be50-02121401ffee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:36:55.083299 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1342/2682] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:36:55 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:36:55.421569 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e4e23e-06e5-44e6-8349-e15fd59087d4 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:36:55.489368 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e4e23e-06e5-44e6-8349-e15fd59087d4 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:36:55.496682 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708410e1-f135-49b2-acee-9f8588622086 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:36:55.496682 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708410e1-f135-49b2-acee-9f8588622086 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:36:55.496682 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708410e1-f135-49b2-acee-9f8588622086 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:36:55.496682 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708410e1-f135-49b2-acee-9f8588622086 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:36:55.497409 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-708410e1-f135-49b2-acee-9f8588622086 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:36:55.508344 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708410e1-f135-49b2-acee-9f8588622086 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:36:55.509847 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708410e1-f135-49b2-acee-9f8588622086 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:36:55.519281 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1341/2683] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:36:55 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:37:00.343441 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7741da7d-f6ba-4560-b2b0-673377d43e19 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:37:00.344007 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7741da7d-f6ba-4560-b2b0-673377d43e19 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:37:00.344007 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7741da7d-f6ba-4560-b2b0-673377d43e19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:37:00.373674 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-7741da7d-f6ba-4560-b2b0-673377d43e19 None None] MFA Rules not processed for user `e080beaac56d42e5ae0f1a4fdea170f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:37:00.375222 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-7741da7d-f6ba-4560-b2b0-673377d43e19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:37:00.490653 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1343/2684] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:37:00 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:37:00.494394 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708410e1-f135-49b2-acee-9f8588622086 service neutron] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:37:00.499931 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-708410e1-f135-49b2-acee-9f8588622086 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:37:00.600908 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708410e1-f135-49b2-acee-9f8588622086 service neutron] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:37:00.605949 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3c3a5d-e341-45dd-9bfd-fd073f2b4ccc service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57e888884d2b4629bfd12d1034ce5be4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:37:00.606928 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3c3a5d-e341-45dd-9bfd-fd073f2b4ccc service glance] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:37:00.607400 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3c3a5d-e341-45dd-9bfd-fd073f2b4ccc service glance] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:37:00.607887 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3c3a5d-e341-45dd-9bfd-fd073f2b4ccc service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:37:00.709650 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3c3a5d-e341-45dd-9bfd-fd073f2b4ccc service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:37:00.711993 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3c3a5d-e341-45dd-9bfd-fd073f2b4ccc service glance] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:37:00.733298 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1342/2685] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:37:00 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 17 19:37:01.034406 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a077173a-57c3-49ce-99fc-a6beddfe13c8 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:37:01.034702 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a077173a-57c3-49ce-99fc-a6beddfe13c8 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:37:01.034978 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a077173a-57c3-49ce-99fc-a6beddfe13c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:37:01.068389 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-a077173a-57c3-49ce-99fc-a6beddfe13c8 None None] MFA Rules not processed for user `2b90b57c542440448579b3aadda8b241`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:37:01.069756 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a077173a-57c3-49ce-99fc-a6beddfe13c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:37:01.184041 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1344/2686] 10.222.0.58 () {58 vars in 1009 bytes} [Tue May 17 19:37:01 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:37:01.365667 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3c3a5d-e341-45dd-9bfd-fd073f2b4ccc service glance] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:37:01.367155 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-6b3c3a5d-e341-45dd-9bfd-fd073f2b4ccc service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:37:01.477458 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3c3a5d-e341-45dd-9bfd-fd073f2b4ccc service glance] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:37:01.483596 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae1761d-8691-41bf-8db3-ea82f03c1077 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0cb292002434d30bc81f370d120a2a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:37:01.484262 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae1761d-8691-41bf-8db3-ea82f03c1077 service swift] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:37:01.484545 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae1761d-8691-41bf-8db3-ea82f03c1077 service swift] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:37:01.484813 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae1761d-8691-41bf-8db3-ea82f03c1077 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:37:01.494384 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae1761d-8691-41bf-8db3-ea82f03c1077 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:37:01.496532 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae1761d-8691-41bf-8db3-ea82f03c1077 service swift] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:37:01.520612 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1343/2687] 10.222.0.58 () {60 vars in 1384 bytes} [Tue May 17 19:37:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 17 19:37:11.861021 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66adee4a-b9d2-498e-aa4d-27092ea5eb8f None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:37:11.861489 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66adee4a-b9d2-498e-aa4d-27092ea5eb8f None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:37:11.861589 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66adee4a-b9d2-498e-aa4d-27092ea5eb8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:37:11.879778 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-66adee4a-b9d2-498e-aa4d-27092ea5eb8f None None] MFA Rules not processed for user `176f09a4dd2349a8aaa45224e8b88023`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:37:11.881322 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-66adee4a-b9d2-498e-aa4d-27092ea5eb8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:37:11.892138 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1345/2688] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:37:11 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:37:15.111788 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6627e02-3516-4c89-8e58-a46a0ff9ae9e None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:37:15.112287 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6627e02-3516-4c89-8e58-a46a0ff9ae9e None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:37:15.112368 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6627e02-3516-4c89-8e58-a46a0ff9ae9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:37:15.130921 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-f6627e02-3516-4c89-8e58-a46a0ff9ae9e None None] MFA Rules not processed for user `2b90b57c542440448579b3aadda8b241`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:37:15.132283 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-f6627e02-3516-4c89-8e58-a46a0ff9ae9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:37:15.139757 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1344/2689] 10.222.0.58 () {58 vars in 1009 bytes} [Tue May 17 19:37:15 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:37:15.152421 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66adee4a-b9d2-498e-aa4d-27092ea5eb8f None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:37:15.164905 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66adee4a-b9d2-498e-aa4d-27092ea5eb8f None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:37:15.170110 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4ef9b5-89bb-43a2-bcfa-aaf12b763c0b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0cb292002434d30bc81f370d120a2a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:37:15.171372 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4ef9b5-89bb-43a2-bcfa-aaf12b763c0b service swift] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:37:15.171694 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4ef9b5-89bb-43a2-bcfa-aaf12b763c0b service swift] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:37:15.171970 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4ef9b5-89bb-43a2-bcfa-aaf12b763c0b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:37:15.180456 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4ef9b5-89bb-43a2-bcfa-aaf12b763c0b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:37:15.182072 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4ef9b5-89bb-43a2-bcfa-aaf12b763c0b service swift] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:37:15.190194 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1346/2690] 10.222.0.58 () {60 vars in 1384 bytes} [Tue May 17 19:37:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 17 19:37:41.435047 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6627e02-3516-4c89-8e58-a46a0ff9ae9e None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:37:41.436470 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-f6627e02-3516-4c89-8e58-a46a0ff9ae9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:37:41.492865 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6627e02-3516-4c89-8e58-a46a0ff9ae9e None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:37:41.496537 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3771a44e-ffad-4ae0-a11a-1a4a2e1ff092 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd05409b38ae346458abfe41ebda1c59e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:37:41.497223 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3771a44e-ffad-4ae0-a11a-1a4a2e1ff092 service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:37:41.497505 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3771a44e-ffad-4ae0-a11a-1a4a2e1ff092 service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:37:41.497774 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3771a44e-ffad-4ae0-a11a-1a4a2e1ff092 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:37:41.500033 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-3771a44e-ffad-4ae0-a11a-1a4a2e1ff092 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:37:41.559250 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3771a44e-ffad-4ae0-a11a-1a4a2e1ff092 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:37:41.560673 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3771a44e-ffad-4ae0-a11a-1a4a2e1ff092 service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:37:41.570433 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1345/2691] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:37:41 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:37:42.105096 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4ef9b5-89bb-43a2-bcfa-aaf12b763c0b service swift] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:37:42.115011 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4ef9b5-89bb-43a2-bcfa-aaf12b763c0b service swift] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:37:42.118708 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e25355a-5f9b-4449-b13d-58d16b434d98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:37:42.119334 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e25355a-5f9b-4449-b13d-58d16b434d98 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:37:42.119652 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e25355a-5f9b-4449-b13d-58d16b434d98 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:37:42.119922 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e25355a-5f9b-4449-b13d-58d16b434d98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:37:42.128673 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e25355a-5f9b-4449-b13d-58d16b434d98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:37:42.130088 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e25355a-5f9b-4449-b13d-58d16b434d98 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:37:42.139119 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1347/2692] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:37:42 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:37:49.896331 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7bc934-61a7-4ae7-904f-5f4d20fd0a10 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:37:49.896877 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7bc934-61a7-4ae7-904f-5f4d20fd0a10 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:37:49.896984 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7bc934-61a7-4ae7-904f-5f4d20fd0a10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:37:49.914266 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-3d7bc934-61a7-4ae7-904f-5f4d20fd0a10 None None] MFA Rules not processed for user `2e4f7fc1ab304418b892be060814bd8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:37:49.915512 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-3d7bc934-61a7-4ae7-904f-5f4d20fd0a10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:37:49.922731 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1346/2693] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:37:49 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:37:50.614551 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e25355a-5f9b-4449-b13d-58d16b434d98 service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:37:50.616225 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-3e25355a-5f9b-4449-b13d-58d16b434d98 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:37:50.685220 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e25355a-5f9b-4449-b13d-58d16b434d98 service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:37:50.691047 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b7a50a-fb39-4953-9032-c5959c51b554 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06538aab5f194af7b7c677d6b94aef2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:37:50.691776 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b7a50a-fb39-4953-9032-c5959c51b554 service placement] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:37:50.692062 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b7a50a-fb39-4953-9032-c5959c51b554 service placement] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:37:50.692344 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b7a50a-fb39-4953-9032-c5959c51b554 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:37:50.694089 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-64b7a50a-fb39-4953-9032-c5959c51b554 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:37:50.706694 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b7a50a-fb39-4953-9032-c5959c51b554 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:37:50.708239 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b7a50a-fb39-4953-9032-c5959c51b554 service placement] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:37:50.718676 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1348/2694] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:37:50 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:39:12.260629 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bd49c9-8d96-44a1-999d-8d47a2120246 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:39:12.261110 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bd49c9-8d96-44a1-999d-8d47a2120246 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:39:12.261213 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bd49c9-8d96-44a1-999d-8d47a2120246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:39:12.279334 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-36bd49c9-8d96-44a1-999d-8d47a2120246 None None] MFA Rules not processed for user `2b90b57c542440448579b3aadda8b241`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:39:12.280571 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-36bd49c9-8d96-44a1-999d-8d47a2120246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:39:12.287745 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1347/2695] 10.222.0.58 () {58 vars in 967 bytes} [Tue May 17 19:39:12 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:39:12.326723 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b7a50a-fb39-4953-9032-c5959c51b554 service placement] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:39:12.337782 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b7a50a-fb39-4953-9032-c5959c51b554 service placement] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:39:12.342827 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d870ce2a-f320-4346-b805-5bbd446c37e0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0cb292002434d30bc81f370d120a2a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:39:12.343450 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d870ce2a-f320-4346-b805-5bbd446c37e0 service swift] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:39:12.343762 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d870ce2a-f320-4346-b805-5bbd446c37e0 service swift] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:39:12.344035 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d870ce2a-f320-4346-b805-5bbd446c37e0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:39:12.351814 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d870ce2a-f320-4346-b805-5bbd446c37e0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:39:12.353384 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d870ce2a-f320-4346-b805-5bbd446c37e0 service swift] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:39:12.362210 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1349/2696] 10.222.0.58 () {60 vars in 1384 bytes} [Tue May 17 19:39:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 17 19:39:21.672965 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bd49c9-8d96-44a1-999d-8d47a2120246 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:39:21.735819 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bd49c9-8d96-44a1-999d-8d47a2120246 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:39:21.740338 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87fe0f7-beb7-48d8-a67a-2bb612d97cbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:39:21.741212 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87fe0f7-beb7-48d8-a67a-2bb612d97cbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:39:21.741487 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87fe0f7-beb7-48d8-a67a-2bb612d97cbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:39:21.741756 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87fe0f7-beb7-48d8-a67a-2bb612d97cbc admin admin] PATH_INFO: `/v3/users/176f09a4dd2349a8aaa45224e8b88023` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:39:21.742921 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87fe0f7-beb7-48d8-a67a-2bb612d97cbc admin admin] RBAC: Authorizing `identity:delete_user(user_id=176f09a4dd2349a8aaa45224e8b88023)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:39:21.744861 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87fe0f7-beb7-48d8-a67a-2bb612d97cbc admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:39:21.964742 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-c87fe0f7-beb7-48d8-a67a-2bb612d97cbc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '176f09a4dd2349a8aaa45224e8b88023'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:39:21.971825 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-c87fe0f7-beb7-48d8-a67a-2bb612d97cbc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '176f09a4dd2349a8aaa45224e8b88023'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:39:21.983901 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-c87fe0f7-beb7-48d8-a67a-2bb612d97cbc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '176f09a4dd2349a8aaa45224e8b88023'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:39:21.984561 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-c87fe0f7-beb7-48d8-a67a-2bb612d97cbc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '176f09a4dd2349a8aaa45224e8b88023'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:39:21.985199 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-c87fe0f7-beb7-48d8-a67a-2bb612d97cbc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '176f09a4dd2349a8aaa45224e8b88023'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:39:22.048861 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1348/2697] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 17 19:39:21 2022] DELETE /identity/v3/users/176f09a4dd2349a8aaa45224e8b88023 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:39:22.308392 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d870ce2a-f320-4346-b805-5bbd446c37e0 service swift] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:39:22.325882 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d870ce2a-f320-4346-b805-5bbd446c37e0 service swift] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:39:22.380315 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e3d0c4-1c04-4d21-acaa-3c5e0918a6c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:39:22.381173 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e3d0c4-1c04-4d21-acaa-3c5e0918a6c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:39:22.381454 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e3d0c4-1c04-4d21-acaa-3c5e0918a6c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:39:22.381730 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e3d0c4-1c04-4d21-acaa-3c5e0918a6c8 admin admin] PATH_INFO: `/v3/projects/de75551df7bb4cadbaf622b4966540fb` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:39:22.382896 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e3d0c4-1c04-4d21-acaa-3c5e0918a6c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=de75551df7bb4cadbaf622b4966540fb)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:39:22.384832 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e3d0c4-1c04-4d21-acaa-3c5e0918a6c8 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:39:22.517046 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-b0e3d0c4-1c04-4d21-acaa-3c5e0918a6c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de75551df7bb4cadbaf622b4966540fb', 'request_id': 'req-b0e3d0c4-1c04-4d21-acaa-3c5e0918a6c8'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:39:22.522324 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-b0e3d0c4-1c04-4d21-acaa-3c5e0918a6c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de75551df7bb4cadbaf622b4966540fb', 'request_id': 'req-b0e3d0c4-1c04-4d21-acaa-3c5e0918a6c8'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:39:22.625880 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-b0e3d0c4-1c04-4d21-acaa-3c5e0918a6c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de75551df7bb4cadbaf622b4966540fb', 'request_id': 'req-b0e3d0c4-1c04-4d21-acaa-3c5e0918a6c8'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:39:22.637115 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-b0e3d0c4-1c04-4d21-acaa-3c5e0918a6c8 admin admin] The token cache is being invalidate because project de75551df7bb4cadbaf622b4966540fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 17 19:39:22.637405 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-b0e3d0c4-1c04-4d21-acaa-3c5e0918a6c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:39:22.638564 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-b0e3d0c4-1c04-4d21-acaa-3c5e0918a6c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:39:22.640271 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1350/2698] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 17 19:39:22 2022] DELETE /identity/v3/projects/de75551df7bb4cadbaf622b4966540fb => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:39:22.644356 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87fe0f7-beb7-48d8-a67a-2bb612d97cbc admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:39:22.645773 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-c87fe0f7-beb7-48d8-a67a-2bb612d97cbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:39:22.658706 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87fe0f7-beb7-48d8-a67a-2bb612d97cbc admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:39:22.712805 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e428fc-b0fb-4ae3-8d05-34b5d3af7790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:39:22.713758 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e428fc-b0fb-4ae3-8d05-34b5d3af7790 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:39:22.713921 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e428fc-b0fb-4ae3-8d05-34b5d3af7790 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:39:22.714187 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e428fc-b0fb-4ae3-8d05-34b5d3af7790 admin admin] PATH_INFO: `/v3/users/e080beaac56d42e5ae0f1a4fdea170f3` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:39:22.715360 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e428fc-b0fb-4ae3-8d05-34b5d3af7790 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e080beaac56d42e5ae0f1a4fdea170f3)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:39:22.717291 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e428fc-b0fb-4ae3-8d05-34b5d3af7790 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:39:22.975867 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-e9e428fc-b0fb-4ae3-8d05-34b5d3af7790 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e080beaac56d42e5ae0f1a4fdea170f3'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:39:22.983195 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-e9e428fc-b0fb-4ae3-8d05-34b5d3af7790 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e080beaac56d42e5ae0f1a4fdea170f3'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:39:22.994256 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-e9e428fc-b0fb-4ae3-8d05-34b5d3af7790 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e080beaac56d42e5ae0f1a4fdea170f3'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:39:22.994898 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-e9e428fc-b0fb-4ae3-8d05-34b5d3af7790 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e080beaac56d42e5ae0f1a4fdea170f3'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:39:22.995504 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-e9e428fc-b0fb-4ae3-8d05-34b5d3af7790 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e080beaac56d42e5ae0f1a4fdea170f3'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:39:23.066044 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1349/2699] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 17 19:39:22 2022] DELETE /identity/v3/users/e080beaac56d42e5ae0f1a4fdea170f3 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:39:23.307278 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e3d0c4-1c04-4d21-acaa-3c5e0918a6c8 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:39:23.313979 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b0e3d0c4-1c04-4d21-acaa-3c5e0918a6c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:39:23.323616 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e3d0c4-1c04-4d21-acaa-3c5e0918a6c8 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:39:23.381348 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7156a0c5-e48f-49e5-8834-0fda1108ffa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:39:23.382253 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7156a0c5-e48f-49e5-8834-0fda1108ffa8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:39:23.382532 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7156a0c5-e48f-49e5-8834-0fda1108ffa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:39:23.382800 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7156a0c5-e48f-49e5-8834-0fda1108ffa8 admin admin] PATH_INFO: `/v3/projects/61f95072dbf14035b151663d3ec2d626` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:39:23.384003 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7156a0c5-e48f-49e5-8834-0fda1108ffa8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=61f95072dbf14035b151663d3ec2d626)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:39:23.385907 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7156a0c5-e48f-49e5-8834-0fda1108ffa8 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:39:23.534903 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-7156a0c5-e48f-49e5-8834-0fda1108ffa8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61f95072dbf14035b151663d3ec2d626', 'request_id': 'req-7156a0c5-e48f-49e5-8834-0fda1108ffa8'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:39:23.540507 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-7156a0c5-e48f-49e5-8834-0fda1108ffa8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61f95072dbf14035b151663d3ec2d626', 'request_id': 'req-7156a0c5-e48f-49e5-8834-0fda1108ffa8'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:39:23.630947 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-7156a0c5-e48f-49e5-8834-0fda1108ffa8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61f95072dbf14035b151663d3ec2d626', 'request_id': 'req-7156a0c5-e48f-49e5-8834-0fda1108ffa8'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:39:23.642769 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-7156a0c5-e48f-49e5-8834-0fda1108ffa8 admin admin] The token cache is being invalidate because project 61f95072dbf14035b151663d3ec2d626 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 17 19:39:23.643060 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-7156a0c5-e48f-49e5-8834-0fda1108ffa8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:39:23.643712 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-7156a0c5-e48f-49e5-8834-0fda1108ffa8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:39:23.645252 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1351/2700] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 17 19:39:23 2022] DELETE /identity/v3/projects/61f95072dbf14035b151663d3ec2d626 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:39:45.460935 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e428fc-b0fb-4ae3-8d05-34b5d3af7790 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:39:45.462525 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-e9e428fc-b0fb-4ae3-8d05-34b5d3af7790 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:39:45.521433 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e428fc-b0fb-4ae3-8d05-34b5d3af7790 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:39:45.568889 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f2c6cd-b263-4f3e-ad6b-85a2f8ddc6df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:39:45.569539 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f2c6cd-b263-4f3e-ad6b-85a2f8ddc6df service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:39:45.569819 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f2c6cd-b263-4f3e-ad6b-85a2f8ddc6df service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:39:45.570091 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f2c6cd-b263-4f3e-ad6b-85a2f8ddc6df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:39:45.572062 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-49f2c6cd-b263-4f3e-ad6b-85a2f8ddc6df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:39:45.640857 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f2c6cd-b263-4f3e-ad6b-85a2f8ddc6df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:39:45.642320 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f2c6cd-b263-4f3e-ad6b-85a2f8ddc6df service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:39:45.653445 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1350/2701] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:39:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:39:46.843048 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7156a0c5-e48f-49e5-8834-0fda1108ffa8 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:39:46.844434 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-7156a0c5-e48f-49e5-8834-0fda1108ffa8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:39:46.857516 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7156a0c5-e48f-49e5-8834-0fda1108ffa8 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:39:46.861293 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca21d289-5f2d-4978-9e1c-b903ace3be49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:39:46.861969 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca21d289-5f2d-4978-9e1c-b903ace3be49 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:39:46.862247 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca21d289-5f2d-4978-9e1c-b903ace3be49 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:39:46.862520 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca21d289-5f2d-4978-9e1c-b903ace3be49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:39:46.864376 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-ca21d289-5f2d-4978-9e1c-b903ace3be49 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:39:46.879147 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca21d289-5f2d-4978-9e1c-b903ace3be49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:39:46.880618 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca21d289-5f2d-4978-9e1c-b903ace3be49 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:39:46.892415 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1352/2702] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:39:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:40:01.351211 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f2c6cd-b263-4f3e-ad6b-85a2f8ddc6df service neutron] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:40:01.352815 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-49f2c6cd-b263-4f3e-ad6b-85a2f8ddc6df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:40:01.364653 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f2c6cd-b263-4f3e-ad6b-85a2f8ddc6df service neutron] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:40:01.368470 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1948b2c3-be50-4558-82f2-c2c68bba2ffd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:40:01.369102 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1948b2c3-be50-4558-82f2-c2c68bba2ffd service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:40:01.369382 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1948b2c3-be50-4558-82f2-c2c68bba2ffd service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:40:01.369657 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1948b2c3-be50-4558-82f2-c2c68bba2ffd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:40:01.371981 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-1948b2c3-be50-4558-82f2-c2c68bba2ffd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:40:01.434828 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1948b2c3-be50-4558-82f2-c2c68bba2ffd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:40:01.436300 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1948b2c3-be50-4558-82f2-c2c68bba2ffd service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:40:01.445928 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1351/2703] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:40:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:40:02.605208 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7864da67-8a5c-4085-aae1-12638ff9442e None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:40:02.606171 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7864da67-8a5c-4085-aae1-12638ff9442e None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:40:02.606171 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7864da67-8a5c-4085-aae1-12638ff9442e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:40:02.624025 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-7864da67-8a5c-4085-aae1-12638ff9442e None None] MFA Rules not processed for user `d823563863d8495f8e0680da4e879439`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:40:02.628629 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-7864da67-8a5c-4085-aae1-12638ff9442e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:40:02.678781 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1353/2704] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:40:02 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:40:08.407044 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1948b2c3-be50-4558-82f2-c2c68bba2ffd service neutron] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:40:08.412662 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-1948b2c3-be50-4558-82f2-c2c68bba2ffd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:40:08.431142 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1948b2c3-be50-4558-82f2-c2c68bba2ffd service neutron] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:40:08.437750 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22895216-b1aa-4c45-a2ba-9b3e8d5674e0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c1f878b0dc4d63a4f5dcb12fac1dc1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:40:08.438497 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22895216-b1aa-4c45-a2ba-9b3e8d5674e0 service nova] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:40:08.438782 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22895216-b1aa-4c45-a2ba-9b3e8d5674e0 service nova] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:40:08.439058 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22895216-b1aa-4c45-a2ba-9b3e8d5674e0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:40:08.441141 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-22895216-b1aa-4c45-a2ba-9b3e8d5674e0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:40:08.454550 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22895216-b1aa-4c45-a2ba-9b3e8d5674e0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:40:08.456194 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22895216-b1aa-4c45-a2ba-9b3e8d5674e0 service nova] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:40:08.475127 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1352/2705] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:40:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:40:09.826176 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7864da67-8a5c-4085-aae1-12638ff9442e None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:40:09.827834 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-7864da67-8a5c-4085-aae1-12638ff9442e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:40:09.842674 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7864da67-8a5c-4085-aae1-12638ff9442e None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:40:09.847176 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a740b1bf-cfae-4ee6-ad27-3cda11c3210b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:40:09.848198 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a740b1bf-cfae-4ee6-ad27-3cda11c3210b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:40:09.848481 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a740b1bf-cfae-4ee6-ad27-3cda11c3210b admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:40:09.848747 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a740b1bf-cfae-4ee6-ad27-3cda11c3210b admin admin] PATH_INFO: `/v3/users/2e4f7fc1ab304418b892be060814bd8f` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:40:09.849923 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a740b1bf-cfae-4ee6-ad27-3cda11c3210b admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e4f7fc1ab304418b892be060814bd8f)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:40:09.851916 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a740b1bf-cfae-4ee6-ad27-3cda11c3210b admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:40:10.021453 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-a740b1bf-cfae-4ee6-ad27-3cda11c3210b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e4f7fc1ab304418b892be060814bd8f'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:40:10.022108 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-a740b1bf-cfae-4ee6-ad27-3cda11c3210b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e4f7fc1ab304418b892be060814bd8f'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:40:10.033607 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-a740b1bf-cfae-4ee6-ad27-3cda11c3210b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e4f7fc1ab304418b892be060814bd8f'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:40:10.122243 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-a740b1bf-cfae-4ee6-ad27-3cda11c3210b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e4f7fc1ab304418b892be060814bd8f'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:40:10.129194 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-a740b1bf-cfae-4ee6-ad27-3cda11c3210b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e4f7fc1ab304418b892be060814bd8f'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:40:10.134974 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1354/2706] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 17 19:40:09 2022] DELETE /identity/v3/users/2e4f7fc1ab304418b892be060814bd8f => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:40:10.369168 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22895216-b1aa-4c45-a2ba-9b3e8d5674e0 service nova] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:40:10.381840 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22895216-b1aa-4c45-a2ba-9b3e8d5674e0 service nova] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:40:10.445543 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d025b80-5512-4f71-8b40-1ea015b2461d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:40:10.465259 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d025b80-5512-4f71-8b40-1ea015b2461d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:40:10.465259 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d025b80-5512-4f71-8b40-1ea015b2461d admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:40:10.465259 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d025b80-5512-4f71-8b40-1ea015b2461d admin admin] PATH_INFO: `/v3/projects/aae36ceec2314da8a829113fc36f2f40` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:40:10.465259 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d025b80-5512-4f71-8b40-1ea015b2461d admin admin] RBAC: Authorizing `identity:delete_project(project_id=aae36ceec2314da8a829113fc36f2f40)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:40:10.465259 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d025b80-5512-4f71-8b40-1ea015b2461d admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:40:10.557773 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-4d025b80-5512-4f71-8b40-1ea015b2461d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aae36ceec2314da8a829113fc36f2f40', 'request_id': 'req-4d025b80-5512-4f71-8b40-1ea015b2461d'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:40:10.564021 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-4d025b80-5512-4f71-8b40-1ea015b2461d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aae36ceec2314da8a829113fc36f2f40', 'request_id': 'req-4d025b80-5512-4f71-8b40-1ea015b2461d'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:40:10.569598 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-4d025b80-5512-4f71-8b40-1ea015b2461d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aae36ceec2314da8a829113fc36f2f40', 'request_id': 'req-4d025b80-5512-4f71-8b40-1ea015b2461d'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:40:10.664140 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-4d025b80-5512-4f71-8b40-1ea015b2461d admin admin] The token cache is being invalidate because project aae36ceec2314da8a829113fc36f2f40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 17 19:40:10.664422 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-4d025b80-5512-4f71-8b40-1ea015b2461d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:40:10.665027 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-4d025b80-5512-4f71-8b40-1ea015b2461d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:40:10.666551 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1353/2707] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 17 19:40:10 2022] DELETE /identity/v3/projects/aae36ceec2314da8a829113fc36f2f40 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:40:10.671111 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a740b1bf-cfae-4ee6-ad27-3cda11c3210b admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:40:10.672496 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a740b1bf-cfae-4ee6-ad27-3cda11c3210b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:40:10.685188 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a740b1bf-cfae-4ee6-ad27-3cda11c3210b admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:40:10.743813 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a51639-05f2-4ad4-a288-08d69d79a1d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:40:10.744688 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a51639-05f2-4ad4-a288-08d69d79a1d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:40:10.744994 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a51639-05f2-4ad4-a288-08d69d79a1d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:40:10.745425 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a51639-05f2-4ad4-a288-08d69d79a1d6 admin admin] PATH_INFO: `/v3/users/61db6d0dfbef43969e2028a24462c360` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:40:10.746652 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a51639-05f2-4ad4-a288-08d69d79a1d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=61db6d0dfbef43969e2028a24462c360)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:40:10.748776 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a51639-05f2-4ad4-a288-08d69d79a1d6 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:40:10.918179 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-08a51639-05f2-4ad4-a288-08d69d79a1d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61db6d0dfbef43969e2028a24462c360'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:40:10.918851 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-08a51639-05f2-4ad4-a288-08d69d79a1d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61db6d0dfbef43969e2028a24462c360'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:40:10.930420 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-08a51639-05f2-4ad4-a288-08d69d79a1d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61db6d0dfbef43969e2028a24462c360'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:40:10.979202 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-08a51639-05f2-4ad4-a288-08d69d79a1d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61db6d0dfbef43969e2028a24462c360'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:40:10.986632 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-08a51639-05f2-4ad4-a288-08d69d79a1d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61db6d0dfbef43969e2028a24462c360'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:40:10.992470 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1355/2708] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 17 19:40:10 2022] DELETE /identity/v3/users/61db6d0dfbef43969e2028a24462c360 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:40:11.220898 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d025b80-5512-4f71-8b40-1ea015b2461d admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:40:11.222300 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-4d025b80-5512-4f71-8b40-1ea015b2461d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:40:11.235757 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d025b80-5512-4f71-8b40-1ea015b2461d admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:40:11.291347 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef248002-dd6b-4fa6-977c-f705237ba8de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:40:11.292223 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef248002-dd6b-4fa6-977c-f705237ba8de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:40:11.292526 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef248002-dd6b-4fa6-977c-f705237ba8de admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:40:11.292761 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef248002-dd6b-4fa6-977c-f705237ba8de admin admin] PATH_INFO: `/v3/projects/701b08622f74425e9cf481c9deebc111` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:40:11.293924 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef248002-dd6b-4fa6-977c-f705237ba8de admin admin] RBAC: Authorizing `identity:delete_project(project_id=701b08622f74425e9cf481c9deebc111)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:40:11.295857 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef248002-dd6b-4fa6-977c-f705237ba8de admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:40:11.455572 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-ef248002-dd6b-4fa6-977c-f705237ba8de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '701b08622f74425e9cf481c9deebc111', 'request_id': 'req-ef248002-dd6b-4fa6-977c-f705237ba8de'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:40:11.462168 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-ef248002-dd6b-4fa6-977c-f705237ba8de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '701b08622f74425e9cf481c9deebc111', 'request_id': 'req-ef248002-dd6b-4fa6-977c-f705237ba8de'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:40:11.467593 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-ef248002-dd6b-4fa6-977c-f705237ba8de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '701b08622f74425e9cf481c9deebc111', 'request_id': 'req-ef248002-dd6b-4fa6-977c-f705237ba8de'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:40:11.572828 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-ef248002-dd6b-4fa6-977c-f705237ba8de admin admin] The token cache is being invalidate because project 701b08622f74425e9cf481c9deebc111 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 17 19:40:11.573128 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-ef248002-dd6b-4fa6-977c-f705237ba8de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:40:11.573743 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-ef248002-dd6b-4fa6-977c-f705237ba8de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:40:11.575390 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1354/2709] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 17 19:40:11 2022] DELETE /identity/v3/projects/701b08622f74425e9cf481c9deebc111 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:40:56.061624 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a51639-05f2-4ad4-a288-08d69d79a1d6 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:40:56.063059 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-08a51639-05f2-4ad4-a288-08d69d79a1d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:40:56.119527 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a51639-05f2-4ad4-a288-08d69d79a1d6 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:40:56.168450 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222644d5-3c0a-4017-b992-ffd7831f7123 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:40:56.169158 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222644d5-3c0a-4017-b992-ffd7831f7123 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:40:56.169435 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222644d5-3c0a-4017-b992-ffd7831f7123 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:40:56.170338 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222644d5-3c0a-4017-b992-ffd7831f7123 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:40:56.172101 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-222644d5-3c0a-4017-b992-ffd7831f7123 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:40:56.181153 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222644d5-3c0a-4017-b992-ffd7831f7123 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:40:56.182502 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222644d5-3c0a-4017-b992-ffd7831f7123 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:40:56.190209 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1356/2710] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:40:56 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:41:32.549835 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef248002-dd6b-4fa6-977c-f705237ba8de admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:41:32.551276 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-ef248002-dd6b-4fa6-977c-f705237ba8de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:41:32.674499 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef248002-dd6b-4fa6-977c-f705237ba8de admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:41:32.679183 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2106af0b-85cc-430c-bc3a-b728dd779a72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:41:32.680066 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2106af0b-85cc-430c-bc3a-b728dd779a72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:41:32.680344 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2106af0b-85cc-430c-bc3a-b728dd779a72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:41:32.680614 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2106af0b-85cc-430c-bc3a-b728dd779a72 admin admin] PATH_INFO: `/v3/users/d823563863d8495f8e0680da4e879439` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:41:32.681803 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2106af0b-85cc-430c-bc3a-b728dd779a72 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d823563863d8495f8e0680da4e879439)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:41:32.683746 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2106af0b-85cc-430c-bc3a-b728dd779a72 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:41:32.916879 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-2106af0b-85cc-430c-bc3a-b728dd779a72 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd823563863d8495f8e0680da4e879439'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:41:32.923420 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-2106af0b-85cc-430c-bc3a-b728dd779a72 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd823563863d8495f8e0680da4e879439'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:41:32.934594 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-2106af0b-85cc-430c-bc3a-b728dd779a72 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd823563863d8495f8e0680da4e879439'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:41:32.935247 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-2106af0b-85cc-430c-bc3a-b728dd779a72 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd823563863d8495f8e0680da4e879439'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:41:32.935853 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-2106af0b-85cc-430c-bc3a-b728dd779a72 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd823563863d8495f8e0680da4e879439'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:41:33.017981 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1355/2711] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 17 19:41:32 2022] DELETE /identity/v3/users/d823563863d8495f8e0680da4e879439 => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:41:33.248135 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222644d5-3c0a-4017-b992-ffd7831f7123 service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:41:33.273567 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222644d5-3c0a-4017-b992-ffd7831f7123 service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:41:33.331844 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c271a91f-ff89-4460-a316-bdf0c727371d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:41:33.332610 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c271a91f-ff89-4460-a316-bdf0c727371d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:41:33.332915 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c271a91f-ff89-4460-a316-bdf0c727371d admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:41:33.333210 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c271a91f-ff89-4460-a316-bdf0c727371d admin admin] PATH_INFO: `/v3/projects/2c9294dbfdbb463baf41d1fea4ae1fbc` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:41:33.334463 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c271a91f-ff89-4460-a316-bdf0c727371d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c9294dbfdbb463baf41d1fea4ae1fbc)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:41:33.336459 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c271a91f-ff89-4460-a316-bdf0c727371d admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:41:33.467443 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c271a91f-ff89-4460-a316-bdf0c727371d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c9294dbfdbb463baf41d1fea4ae1fbc', 'request_id': 'req-c271a91f-ff89-4460-a316-bdf0c727371d'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:41:33.473531 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c271a91f-ff89-4460-a316-bdf0c727371d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c9294dbfdbb463baf41d1fea4ae1fbc', 'request_id': 'req-c271a91f-ff89-4460-a316-bdf0c727371d'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:41:33.566192 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c271a91f-ff89-4460-a316-bdf0c727371d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c9294dbfdbb463baf41d1fea4ae1fbc', 'request_id': 'req-c271a91f-ff89-4460-a316-bdf0c727371d'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:41:33.577484 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c271a91f-ff89-4460-a316-bdf0c727371d admin admin] The token cache is being invalidate because project 2c9294dbfdbb463baf41d1fea4ae1fbc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 17 19:41:33.577769 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c271a91f-ff89-4460-a316-bdf0c727371d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:41:33.578375 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c271a91f-ff89-4460-a316-bdf0c727371d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:41:33.580516 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1357/2712] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 17 19:41:33 2022] DELETE /identity/v3/projects/2c9294dbfdbb463baf41d1fea4ae1fbc => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:41:33.585675 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cd4537-dc11-4263-bdc9-319c296d5933 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:41:33.586162 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cd4537-dc11-4263-bdc9-319c296d5933 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:41:33.586162 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cd4537-dc11-4263-bdc9-319c296d5933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:41:33.634810 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-52cd4537-dc11-4263-bdc9-319c296d5933 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:41:33.636021 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-52cd4537-dc11-4263-bdc9-319c296d5933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:41:33.691764 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1356/2713] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:41:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 17 19:41:33.699905 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d54c171-2c43-4fd3-9020-5cf8c9921beb None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:41:33.700198 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d54c171-2c43-4fd3-9020-5cf8c9921beb None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:41:33.700478 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d54c171-2c43-4fd3-9020-5cf8c9921beb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:41:33.718302 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-9d54c171-2c43-4fd3-9020-5cf8c9921beb None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:41:33.719511 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-9d54c171-2c43-4fd3-9020-5cf8c9921beb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:41:33.727855 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1358/2714] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:41:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:41:33.732272 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cd4537-dc11-4263-bdc9-319c296d5933 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:41:33.742739 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cd4537-dc11-4263-bdc9-319c296d5933 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:41:33.747465 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160b7ee0-3966-4cfd-b48b-e4d1ce5589af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:41:33.748159 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160b7ee0-3966-4cfd-b48b-e4d1ce5589af admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:41:33.748440 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160b7ee0-3966-4cfd-b48b-e4d1ce5589af admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:41:33.748713 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160b7ee0-3966-4cfd-b48b-e4d1ce5589af admin admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:41:33.749431 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160b7ee0-3966-4cfd-b48b-e4d1ce5589af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:41:33.749731 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160b7ee0-3966-4cfd-b48b-e4d1ce5589af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:41:33.751337 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160b7ee0-3966-4cfd-b48b-e4d1ce5589af admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:41:33.762545 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1357/2715] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 17 19:41:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:41:33.766763 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d54c171-2c43-4fd3-9020-5cf8c9921beb None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:41:33.776010 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d54c171-2c43-4fd3-9020-5cf8c9921beb None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:41:33.780661 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64532c9b-2905-4da8-bbcf-f6830568ea76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:41:33.781301 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64532c9b-2905-4da8-bbcf-f6830568ea76 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:41:33.781575 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64532c9b-2905-4da8-bbcf-f6830568ea76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:41:33.781846 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64532c9b-2905-4da8-bbcf-f6830568ea76 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:41:33.783047 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64532c9b-2905-4da8-bbcf-f6830568ea76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:41:33.784984 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64532c9b-2905-4da8-bbcf-f6830568ea76 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:41:33.864774 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1359/2716] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 17 19:41:33 2022] POST /identity/v3/projects => generated 385 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:41:33.868645 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160b7ee0-3966-4cfd-b48b-e4d1ce5589af admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:41:33.878319 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160b7ee0-3966-4cfd-b48b-e4d1ce5589af admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:41:33.932285 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81204215-cb62-4789-afb5-9c57ac0accd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:41:33.932929 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81204215-cb62-4789-afb5-9c57ac0accd1 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:41:33.933204 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81204215-cb62-4789-afb5-9c57ac0accd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:41:33.933472 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81204215-cb62-4789-afb5-9c57ac0accd1 admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:41:33.934130 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81204215-cb62-4789-afb5-9c57ac0accd1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:41:33.936055 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81204215-cb62-4789-afb5-9c57ac0accd1 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:41:34.040596 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1358/2717] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 17 19:41:33 2022] POST /identity/v3/users => generated 293 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:41:34.044518 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64532c9b-2905-4da8-bbcf-f6830568ea76 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:41:34.054666 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64532c9b-2905-4da8-bbcf-f6830568ea76 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:41:34.059292 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f15de1-d038-4e4f-9d8a-b5df8288c154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:41:34.059979 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f15de1-d038-4e4f-9d8a-b5df8288c154 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:41:34.060257 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f15de1-d038-4e4f-9d8a-b5df8288c154 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:41:34.060530 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f15de1-d038-4e4f-9d8a-b5df8288c154 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:41:34.061275 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f15de1-d038-4e4f-9d8a-b5df8288c154 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:41:34.062908 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f15de1-d038-4e4f-9d8a-b5df8288c154 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:41:34.074248 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1360/2718] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:41:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:41:34.077939 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81204215-cb62-4789-afb5-9c57ac0accd1 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:41:34.088127 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81204215-cb62-4789-afb5-9c57ac0accd1 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:41:34.092732 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4459aa66-346e-4997-a8fe-91e2fdd2fce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:41:34.093372 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4459aa66-346e-4997-a8fe-91e2fdd2fce6 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:41:34.093650 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4459aa66-346e-4997-a8fe-91e2fdd2fce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:41:34.093926 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4459aa66-346e-4997-a8fe-91e2fdd2fce6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:41:34.094634 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4459aa66-346e-4997-a8fe-91e2fdd2fce6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:41:34.096904 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4459aa66-346e-4997-a8fe-91e2fdd2fce6 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:41:34.106653 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1359/2719] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:41:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:41:34.110752 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f15de1-d038-4e4f-9d8a-b5df8288c154 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:41:34.120269 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f15de1-d038-4e4f-9d8a-b5df8288c154 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:41:34.124669 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3349c261-2591-49f7-9f78-e8938179a65b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:41:34.125513 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3349c261-2591-49f7-9f78-e8938179a65b admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:41:34.125784 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3349c261-2591-49f7-9f78-e8938179a65b admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:41:34.126052 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3349c261-2591-49f7-9f78-e8938179a65b admin admin] PATH_INFO: `/v3/projects/e1e2d6671a8b45c6a698b4ef0263f480/users/3e618c109a9a4de9b79842fb7fb19947/roles/114e6a16e63447d1924707ace5aace25` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:41:34.139962 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3349c261-2591-49f7-9f78-e8938179a65b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1e2d6671a8b45c6a698b4ef0263f480, user_id=3e618c109a9a4de9b79842fb7fb19947, role_id=114e6a16e63447d1924707ace5aace25)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:41:34.143104 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3349c261-2591-49f7-9f78-e8938179a65b admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:41:34.234758 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1361/2720] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:41:34 2022] PUT /identity/v3/projects/e1e2d6671a8b45c6a698b4ef0263f480/users/3e618c109a9a4de9b79842fb7fb19947/roles/114e6a16e63447d1924707ace5aace25 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:41:34.239804 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10afe9f7-38ea-40ac-b6cd-c2286d0dacd4 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:41:34.240091 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10afe9f7-38ea-40ac-b6cd-c2286d0dacd4 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:41:34.240364 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10afe9f7-38ea-40ac-b6cd-c2286d0dacd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:41:34.258065 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-10afe9f7-38ea-40ac-b6cd-c2286d0dacd4 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:41:34.259236 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-10afe9f7-38ea-40ac-b6cd-c2286d0dacd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:41:34.314961 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1360/2721] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:41:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:41:34.324339 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3349c261-2591-49f7-9f78-e8938179a65b admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:41:34.326014 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-3349c261-2591-49f7-9f78-e8938179a65b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:41:34.415541 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3349c261-2591-49f7-9f78-e8938179a65b admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:41:34.419146 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77555edb-b93c-4e8d-947a-e28a234094df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:41:34.419814 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77555edb-b93c-4e8d-947a-e28a234094df service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:41:34.420089 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77555edb-b93c-4e8d-947a-e28a234094df service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:41:34.420356 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77555edb-b93c-4e8d-947a-e28a234094df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:41:34.429104 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77555edb-b93c-4e8d-947a-e28a234094df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:41:34.430522 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77555edb-b93c-4e8d-947a-e28a234094df service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:41:34.440932 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1362/2722] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:41:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:41:39.615402 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a65bd07-3d31-4b13-b1fb-df3205a7732c None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:41:39.616032 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a65bd07-3d31-4b13-b1fb-df3205a7732c None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:41:39.616032 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a65bd07-3d31-4b13-b1fb-df3205a7732c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:41:39.645322 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-1a65bd07-3d31-4b13-b1fb-df3205a7732c None None] MFA Rules not processed for user `3e618c109a9a4de9b79842fb7fb19947`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:41:39.646690 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-1a65bd07-3d31-4b13-b1fb-df3205a7732c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:41:39.715260 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1361/2723] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:41:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:41:39.734333 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77555edb-b93c-4e8d-947a-e28a234094df service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:41:39.736438 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-77555edb-b93c-4e8d-947a-e28a234094df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:41:39.811978 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77555edb-b93c-4e8d-947a-e28a234094df service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:41:39.817671 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440a12bc-b656-4e94-bb08-8e853a952c5a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c1f878b0dc4d63a4f5dcb12fac1dc1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:41:39.818321 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440a12bc-b656-4e94-bb08-8e853a952c5a service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:41:39.818600 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440a12bc-b656-4e94-bb08-8e853a952c5a service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:41:39.818878 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440a12bc-b656-4e94-bb08-8e853a952c5a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:41:39.827359 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440a12bc-b656-4e94-bb08-8e853a952c5a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:41:39.829540 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440a12bc-b656-4e94-bb08-8e853a952c5a service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:41:39.838544 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1363/2724] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:41:39 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:41:42.105697 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236dba60-a655-4890-ad85-6f1303089cb3 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:41:42.106237 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236dba60-a655-4890-ad85-6f1303089cb3 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:41:42.106237 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236dba60-a655-4890-ad85-6f1303089cb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:41:42.124958 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-236dba60-a655-4890-ad85-6f1303089cb3 None None] MFA Rules not processed for user `3e618c109a9a4de9b79842fb7fb19947`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:41:42.126791 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-236dba60-a655-4890-ad85-6f1303089cb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:41:42.134096 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1362/2725] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:41:42 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:41:42.474929 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440a12bc-b656-4e94-bb08-8e853a952c5a service nova] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:41:42.485820 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440a12bc-b656-4e94-bb08-8e853a952c5a service nova] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:41:42.489762 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f395bcf5-353c-4520-9fe8-b7fb4095ad78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:41:42.490402 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f395bcf5-353c-4520-9fe8-b7fb4095ad78 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:41:42.490684 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f395bcf5-353c-4520-9fe8-b7fb4095ad78 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:41:42.490954 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f395bcf5-353c-4520-9fe8-b7fb4095ad78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:41:42.493371 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-f395bcf5-353c-4520-9fe8-b7fb4095ad78 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:41:42.506118 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f395bcf5-353c-4520-9fe8-b7fb4095ad78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:41:42.506118 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f395bcf5-353c-4520-9fe8-b7fb4095ad78 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:41:42.514169 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1364/2726] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:41:42 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:41:43.228330 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236dba60-a655-4890-ad85-6f1303089cb3 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:41:43.229795 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-236dba60-a655-4890-ad85-6f1303089cb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:41:43.305662 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236dba60-a655-4890-ad85-6f1303089cb3 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:41:43.310613 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29933c73-c579-4b4c-aed6-ac03d7231423 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06538aab5f194af7b7c677d6b94aef2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:41:43.314286 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29933c73-c579-4b4c-aed6-ac03d7231423 service placement] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:41:43.314286 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29933c73-c579-4b4c-aed6-ac03d7231423 service placement] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:41:43.314286 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29933c73-c579-4b4c-aed6-ac03d7231423 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:41:43.314286 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-29933c73-c579-4b4c-aed6-ac03d7231423 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:41:43.326144 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29933c73-c579-4b4c-aed6-ac03d7231423 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:41:43.327763 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29933c73-c579-4b4c-aed6-ac03d7231423 service placement] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:41:43.338779 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1363/2727] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:41:43 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:41:52.560677 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f395bcf5-353c-4520-9fe8-b7fb4095ad78 service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:41:52.580857 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f395bcf5-353c-4520-9fe8-b7fb4095ad78 service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:41:52.586272 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074d5917-0351-4146-871c-6a2101705467 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c1f878b0dc4d63a4f5dcb12fac1dc1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:41:52.586901 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074d5917-0351-4146-871c-6a2101705467 service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:41:52.587200 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074d5917-0351-4146-871c-6a2101705467 service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:41:52.587471 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074d5917-0351-4146-871c-6a2101705467 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:41:52.589274 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-074d5917-0351-4146-871c-6a2101705467 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:41:52.601515 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074d5917-0351-4146-871c-6a2101705467 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:41:52.603179 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074d5917-0351-4146-871c-6a2101705467 service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:41:52.613702 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1365/2728] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:41:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:41:55.771897 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29933c73-c579-4b4c-aed6-ac03d7231423 service placement] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:41:55.786980 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29933c73-c579-4b4c-aed6-ac03d7231423 service placement] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:41:55.792850 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c342b230-98e8-453f-9ebd-15c5acebe151 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c1f878b0dc4d63a4f5dcb12fac1dc1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:41:55.793539 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c342b230-98e8-453f-9ebd-15c5acebe151 service nova] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:41:55.794630 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c342b230-98e8-453f-9ebd-15c5acebe151 service nova] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:41:55.794630 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c342b230-98e8-453f-9ebd-15c5acebe151 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:41:55.796069 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-c342b230-98e8-453f-9ebd-15c5acebe151 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:41:55.809502 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c342b230-98e8-453f-9ebd-15c5acebe151 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:41:55.811149 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c342b230-98e8-453f-9ebd-15c5acebe151 service nova] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:41:55.823309 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1364/2729] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:41:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:42:43.828710 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02efb6d8-4f96-4c6d-9074-35f286048715 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:42:43.829200 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02efb6d8-4f96-4c6d-9074-35f286048715 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:42:43.829301 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02efb6d8-4f96-4c6d-9074-35f286048715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:42:43.847246 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-02efb6d8-4f96-4c6d-9074-35f286048715 None None] MFA Rules not processed for user `3e618c109a9a4de9b79842fb7fb19947`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:42:43.848460 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-02efb6d8-4f96-4c6d-9074-35f286048715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:42:43.856146 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1366/2730] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:42:43 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:42:44.151628 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c342b230-98e8-453f-9ebd-15c5acebe151 service nova] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:42:44.153016 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-c342b230-98e8-453f-9ebd-15c5acebe151 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:42:44.164065 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c342b230-98e8-453f-9ebd-15c5acebe151 service nova] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:42:44.167627 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac6b16c-4efa-4b43-b600-c018c6804965 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:42:44.168247 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac6b16c-4efa-4b43-b600-c018c6804965 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:42:44.168526 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac6b16c-4efa-4b43-b600-c018c6804965 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:42:44.168797 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac6b16c-4efa-4b43-b600-c018c6804965 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:42:44.170463 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-5ac6b16c-4efa-4b43-b600-c018c6804965 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:42:44.179927 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac6b16c-4efa-4b43-b600-c018c6804965 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:42:44.181325 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac6b16c-4efa-4b43-b600-c018c6804965 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:42:44.189038 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1365/2731] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:42:44 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:42:50.893414 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02efb6d8-4f96-4c6d-9074-35f286048715 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:42:50.905176 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02efb6d8-4f96-4c6d-9074-35f286048715 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:42:50.910116 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaa041e-7453-4118-b673-1b76f29d850c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06538aab5f194af7b7c677d6b94aef2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:42:50.910745 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaa041e-7453-4118-b673-1b76f29d850c service placement] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:42:50.911025 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaa041e-7453-4118-b673-1b76f29d850c service placement] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:42:50.911300 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaa041e-7453-4118-b673-1b76f29d850c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:42:50.913062 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-acaa041e-7453-4118-b673-1b76f29d850c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:42:50.936624 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaa041e-7453-4118-b673-1b76f29d850c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:42:50.938230 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaa041e-7453-4118-b673-1b76f29d850c service placement] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:42:50.957833 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1367/2732] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:42:50 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:43:54.016334 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088d4cfb-1313-4111-99bb-bc4506df3347 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:43:54.016845 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088d4cfb-1313-4111-99bb-bc4506df3347 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:43:54.016958 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088d4cfb-1313-4111-99bb-bc4506df3347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:43:54.034763 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-088d4cfb-1313-4111-99bb-bc4506df3347 None None] MFA Rules not processed for user `3e618c109a9a4de9b79842fb7fb19947`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:43:54.036055 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-088d4cfb-1313-4111-99bb-bc4506df3347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:43:54.043566 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1366/2733] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:43:54 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 17 19:44:48.179674 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaa041e-7453-4118-b673-1b76f29d850c service placement] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:44:48.181044 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-acaa041e-7453-4118-b673-1b76f29d850c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:44:48.254473 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaa041e-7453-4118-b673-1b76f29d850c service placement] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:44:48.258053 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eede3403-e12f-4526-9dd0-4da4067b02d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd05409b38ae346458abfe41ebda1c59e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:44:48.258714 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eede3403-e12f-4526-9dd0-4da4067b02d1 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:44:48.258988 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eede3403-e12f-4526-9dd0-4da4067b02d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:44:48.259254 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eede3403-e12f-4526-9dd0-4da4067b02d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:44:48.261000 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-eede3403-e12f-4526-9dd0-4da4067b02d1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:44:48.285888 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eede3403-e12f-4526-9dd0-4da4067b02d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:44:48.287521 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eede3403-e12f-4526-9dd0-4da4067b02d1 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:44:48.308336 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1368/2734] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:44:48 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 17 19:44:56.656685 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088d4cfb-1313-4111-99bb-bc4506df3347 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:44:56.666807 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088d4cfb-1313-4111-99bb-bc4506df3347 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:44:56.670377 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db79183-b647-4cd5-bad9-5b84803e17a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd05409b38ae346458abfe41ebda1c59e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:44:56.671019 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db79183-b647-4cd5-bad9-5b84803e17a6 service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:44:56.671283 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db79183-b647-4cd5-bad9-5b84803e17a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:44:56.671594 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db79183-b647-4cd5-bad9-5b84803e17a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:44:56.673354 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-9db79183-b647-4cd5-bad9-5b84803e17a6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:44:56.686739 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db79183-b647-4cd5-bad9-5b84803e17a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:44:56.688199 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db79183-b647-4cd5-bad9-5b84803e17a6 service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:44:56.699559 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1367/2735] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:44:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:45:06.391794 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94080f2-2266-4bfd-b8e1-ea65a77af5ec None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:45:06.392336 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94080f2-2266-4bfd-b8e1-ea65a77af5ec None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:45:06.392336 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94080f2-2266-4bfd-b8e1-ea65a77af5ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:45:06.409292 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-e94080f2-2266-4bfd-b8e1-ea65a77af5ec None None] MFA Rules not processed for user `3e618c109a9a4de9b79842fb7fb19947`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:45:06.410463 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-e94080f2-2266-4bfd-b8e1-ea65a77af5ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:45:06.417899 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1369/2736] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:45:06 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:45:10.891201 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db79183-b647-4cd5-bad9-5b84803e17a6 service cinder] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:45:10.904886 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db79183-b647-4cd5-bad9-5b84803e17a6 service cinder] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:45:10.910764 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306d2c0b-4def-4391-93c7-5dcd581f2e38 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c1f878b0dc4d63a4f5dcb12fac1dc1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:45:10.914550 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306d2c0b-4def-4391-93c7-5dcd581f2e38 service nova] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:45:10.914550 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306d2c0b-4def-4391-93c7-5dcd581f2e38 service nova] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:45:10.914550 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306d2c0b-4def-4391-93c7-5dcd581f2e38 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:45:10.914550 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-306d2c0b-4def-4391-93c7-5dcd581f2e38 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:45:10.931713 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306d2c0b-4def-4391-93c7-5dcd581f2e38 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:45:10.933535 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306d2c0b-4def-4391-93c7-5dcd581f2e38 service nova] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:45:10.945254 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1368/2737] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:45:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:45:58.026888 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94080f2-2266-4bfd-b8e1-ea65a77af5ec None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:45:58.036431 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94080f2-2266-4bfd-b8e1-ea65a77af5ec None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:45:58.040122 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ce9fa5-00c4-4a82-a144-7a7ba30a2c22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:45:58.046927 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ce9fa5-00c4-4a82-a144-7a7ba30a2c22 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:45:58.046927 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ce9fa5-00c4-4a82-a144-7a7ba30a2c22 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:45:58.046927 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ce9fa5-00c4-4a82-a144-7a7ba30a2c22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:45:58.046927 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-46ce9fa5-00c4-4a82-a144-7a7ba30a2c22 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:45:58.052891 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ce9fa5-00c4-4a82-a144-7a7ba30a2c22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:45:58.054308 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ce9fa5-00c4-4a82-a144-7a7ba30a2c22 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:45:58.062885 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1370/2738] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:45:58 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:46:21.801258 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306d2c0b-4def-4391-93c7-5dcd581f2e38 service nova] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:46:21.811743 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306d2c0b-4def-4391-93c7-5dcd581f2e38 service nova] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:46:21.816348 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dc057e-96d0-4202-9373-ef6c3817be67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:46:21.817200 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dc057e-96d0-4202-9373-ef6c3817be67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:46:21.817479 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dc057e-96d0-4202-9373-ef6c3817be67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:46:21.817751 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dc057e-96d0-4202-9373-ef6c3817be67 admin admin] PATH_INFO: `/v3/users/3e618c109a9a4de9b79842fb7fb19947` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:46:21.818929 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dc057e-96d0-4202-9373-ef6c3817be67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e618c109a9a4de9b79842fb7fb19947)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:46:21.820871 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dc057e-96d0-4202-9373-ef6c3817be67 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:46:22.041976 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-a3dc057e-96d0-4202-9373-ef6c3817be67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e618c109a9a4de9b79842fb7fb19947'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:46:22.048691 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-a3dc057e-96d0-4202-9373-ef6c3817be67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e618c109a9a4de9b79842fb7fb19947'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:46:22.060049 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-a3dc057e-96d0-4202-9373-ef6c3817be67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e618c109a9a4de9b79842fb7fb19947'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:46:22.060725 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-a3dc057e-96d0-4202-9373-ef6c3817be67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e618c109a9a4de9b79842fb7fb19947'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:46:22.061346 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-a3dc057e-96d0-4202-9373-ef6c3817be67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e618c109a9a4de9b79842fb7fb19947'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:46:22.188011 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1369/2739] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 17 19:46:21 2022] DELETE /identity/v3/users/3e618c109a9a4de9b79842fb7fb19947 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:46:22.459212 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ce9fa5-00c4-4a82-a144-7a7ba30a2c22 service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:46:22.470547 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ce9fa5-00c4-4a82-a144-7a7ba30a2c22 service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:46:22.525414 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd98875b-938a-4509-87ff-34cb6e779600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:46:22.526270 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd98875b-938a-4509-87ff-34cb6e779600 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:46:22.526548 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd98875b-938a-4509-87ff-34cb6e779600 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:46:22.526818 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd98875b-938a-4509-87ff-34cb6e779600 admin admin] PATH_INFO: `/v3/projects/e1e2d6671a8b45c6a698b4ef0263f480` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:46:22.528016 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd98875b-938a-4509-87ff-34cb6e779600 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1e2d6671a8b45c6a698b4ef0263f480)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:46:22.529961 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd98875b-938a-4509-87ff-34cb6e779600 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:46:22.628406 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-dd98875b-938a-4509-87ff-34cb6e779600 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1e2d6671a8b45c6a698b4ef0263f480', 'request_id': 'req-dd98875b-938a-4509-87ff-34cb6e779600'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:46:22.633517 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-dd98875b-938a-4509-87ff-34cb6e779600 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1e2d6671a8b45c6a698b4ef0263f480', 'request_id': 'req-dd98875b-938a-4509-87ff-34cb6e779600'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:46:22.708568 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-dd98875b-938a-4509-87ff-34cb6e779600 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1e2d6671a8b45c6a698b4ef0263f480', 'request_id': 'req-dd98875b-938a-4509-87ff-34cb6e779600'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:46:22.719937 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-dd98875b-938a-4509-87ff-34cb6e779600 admin admin] The token cache is being invalidate because project e1e2d6671a8b45c6a698b4ef0263f480 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 17 19:46:22.720219 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-dd98875b-938a-4509-87ff-34cb6e779600 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:46:22.720841 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-dd98875b-938a-4509-87ff-34cb6e779600 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:46:22.722368 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1371/2740] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 17 19:46:22 2022] DELETE /identity/v3/projects/e1e2d6671a8b45c6a698b4ef0263f480 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:46:22.727192 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730f3afa-2e99-4bbe-982b-cb2ad37fe02b None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:46:22.727476 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730f3afa-2e99-4bbe-982b-cb2ad37fe02b None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:46:22.727799 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730f3afa-2e99-4bbe-982b-cb2ad37fe02b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:46:22.755814 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-730f3afa-2e99-4bbe-982b-cb2ad37fe02b None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:46:22.756991 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-730f3afa-2e99-4bbe-982b-cb2ad37fe02b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:46:22.814222 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1370/2741] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:46:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:46:22.822437 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481c91f8-9985-43ed-ad98-4cf2f891ad9e None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:46:22.822722 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481c91f8-9985-43ed-ad98-4cf2f891ad9e None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:46:22.822997 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481c91f8-9985-43ed-ad98-4cf2f891ad9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:46:22.840233 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-481c91f8-9985-43ed-ad98-4cf2f891ad9e None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:46:22.841479 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-481c91f8-9985-43ed-ad98-4cf2f891ad9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:46:22.849311 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1372/2742] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:46:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:46:22.853339 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730f3afa-2e99-4bbe-982b-cb2ad37fe02b None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:46:22.863727 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730f3afa-2e99-4bbe-982b-cb2ad37fe02b None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:46:22.868221 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdb297d-37db-412c-9cba-79a995d85735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:46:22.868858 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdb297d-37db-412c-9cba-79a995d85735 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:46:22.869130 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdb297d-37db-412c-9cba-79a995d85735 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:46:22.869400 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdb297d-37db-412c-9cba-79a995d85735 admin admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:46:22.870110 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdb297d-37db-412c-9cba-79a995d85735 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:46:22.870437 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdb297d-37db-412c-9cba-79a995d85735 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:46:22.872725 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdb297d-37db-412c-9cba-79a995d85735 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:46:22.884230 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1371/2743] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 17 19:46:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:46:22.888345 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481c91f8-9985-43ed-ad98-4cf2f891ad9e None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:46:22.897403 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481c91f8-9985-43ed-ad98-4cf2f891ad9e None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:46:22.902029 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78cfa57-6bb5-4c5a-bc47-b66d6712aee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:46:22.902668 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78cfa57-6bb5-4c5a-bc47-b66d6712aee0 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:46:22.902947 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78cfa57-6bb5-4c5a-bc47-b66d6712aee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:46:22.903219 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78cfa57-6bb5-4c5a-bc47-b66d6712aee0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:46:22.903942 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78cfa57-6bb5-4c5a-bc47-b66d6712aee0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:46:22.905840 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78cfa57-6bb5-4c5a-bc47-b66d6712aee0 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:46:23.019579 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1373/2744] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 17 19:46:22 2022] POST /identity/v3/projects => generated 355 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:46:23.023090 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdb297d-37db-412c-9cba-79a995d85735 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:46:23.033003 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdb297d-37db-412c-9cba-79a995d85735 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:46:23.088508 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e1b737-1f2c-400b-91d2-becf82c46609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:46:23.089178 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e1b737-1f2c-400b-91d2-becf82c46609 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:46:23.089446 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e1b737-1f2c-400b-91d2-becf82c46609 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:46:23.089717 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e1b737-1f2c-400b-91d2-becf82c46609 admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:46:23.090401 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e1b737-1f2c-400b-91d2-becf82c46609 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:46:23.092360 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e1b737-1f2c-400b-91d2-becf82c46609 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:46:23.199782 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1372/2745] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 17 19:46:23 2022] POST /identity/v3/users => generated 278 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:46:23.203837 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78cfa57-6bb5-4c5a-bc47-b66d6712aee0 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:46:23.213997 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78cfa57-6bb5-4c5a-bc47-b66d6712aee0 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:46:23.219233 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25924a4f-85ff-4e18-999b-5973d8117bd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:46:23.219919 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25924a4f-85ff-4e18-999b-5973d8117bd2 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:46:23.220199 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25924a4f-85ff-4e18-999b-5973d8117bd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:46:23.220475 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25924a4f-85ff-4e18-999b-5973d8117bd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:46:23.221191 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25924a4f-85ff-4e18-999b-5973d8117bd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:46:23.222847 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25924a4f-85ff-4e18-999b-5973d8117bd2 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:46:23.233530 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1374/2746] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:46:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:46:23.243660 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e1b737-1f2c-400b-91d2-becf82c46609 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:46:23.253462 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e1b737-1f2c-400b-91d2-becf82c46609 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:46:23.258066 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808f77d1-8490-4dd4-b91b-ac123bf21976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:46:23.258710 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808f77d1-8490-4dd4-b91b-ac123bf21976 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:46:23.258993 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808f77d1-8490-4dd4-b91b-ac123bf21976 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:46:23.259263 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808f77d1-8490-4dd4-b91b-ac123bf21976 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:46:23.260005 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808f77d1-8490-4dd4-b91b-ac123bf21976 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:46:23.261592 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808f77d1-8490-4dd4-b91b-ac123bf21976 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:46:23.271188 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1373/2747] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:46:23 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:46:23.275162 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25924a4f-85ff-4e18-999b-5973d8117bd2 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:46:23.285386 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25924a4f-85ff-4e18-999b-5973d8117bd2 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:46:23.289856 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4b28fa-bd19-49e5-8567-f63ae30d2c21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:46:23.290706 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4b28fa-bd19-49e5-8567-f63ae30d2c21 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:46:23.290977 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4b28fa-bd19-49e5-8567-f63ae30d2c21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:46:23.291246 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4b28fa-bd19-49e5-8567-f63ae30d2c21 admin admin] PATH_INFO: `/v3/projects/a0d9a07571374003a84ae4703d71d7a8/users/d92c85679c004cd7a758d82ba12e7d39/roles/114e6a16e63447d1924707ace5aace25` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:46:23.305777 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4b28fa-bd19-49e5-8567-f63ae30d2c21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0d9a07571374003a84ae4703d71d7a8, user_id=d92c85679c004cd7a758d82ba12e7d39, role_id=114e6a16e63447d1924707ace5aace25)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:46:23.308476 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4b28fa-bd19-49e5-8567-f63ae30d2c21 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:46:23.388467 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1375/2748] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:46:23 2022] PUT /identity/v3/projects/a0d9a07571374003a84ae4703d71d7a8/users/d92c85679c004cd7a758d82ba12e7d39/roles/114e6a16e63447d1924707ace5aace25 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 17 19:46:23.398268 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42b8106-b338-44ec-a160-ded2ec447a0a None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:46:23.398553 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42b8106-b338-44ec-a160-ded2ec447a0a None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:46:23.398880 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42b8106-b338-44ec-a160-ded2ec447a0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:46:23.425645 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-e42b8106-b338-44ec-a160-ded2ec447a0a None None] MFA Rules not processed for user `d92c85679c004cd7a758d82ba12e7d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:46:23.426831 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-e42b8106-b338-44ec-a160-ded2ec447a0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:46:23.487981 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1374/2749] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:46:23 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:46:23.492259 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4b28fa-bd19-49e5-8567-f63ae30d2c21 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:46:23.551882 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4b28fa-bd19-49e5-8567-f63ae30d2c21 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:46:23.556492 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bc863b-2530-4121-a570-5d45dd7d83ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:46:23.557144 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bc863b-2530-4121-a570-5d45dd7d83ff admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:46:23.557419 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bc863b-2530-4121-a570-5d45dd7d83ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:46:23.557689 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bc863b-2530-4121-a570-5d45dd7d83ff admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:46:23.558406 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bc863b-2530-4121-a570-5d45dd7d83ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:46:23.560330 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bc863b-2530-4121-a570-5d45dd7d83ff admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:46:23.738730 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1376/2750] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 17 19:46:23 2022] POST /identity/v3/projects => generated 355 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:46:23.742687 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42b8106-b338-44ec-a160-ded2ec447a0a None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:46:23.752958 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42b8106-b338-44ec-a160-ded2ec447a0a None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:46:23.806299 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e5dde8-5b37-4daf-9713-2099e1fc5abb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:46:23.806958 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e5dde8-5b37-4daf-9713-2099e1fc5abb admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:46:23.807239 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e5dde8-5b37-4daf-9713-2099e1fc5abb admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:46:23.807535 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e5dde8-5b37-4daf-9713-2099e1fc5abb admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:46:23.808210 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e5dde8-5b37-4daf-9713-2099e1fc5abb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:46:23.810103 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e5dde8-5b37-4daf-9713-2099e1fc5abb admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:46:23.931781 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1375/2751] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 17 19:46:23 2022] POST /identity/v3/users => generated 284 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:46:23.935659 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bc863b-2530-4121-a570-5d45dd7d83ff admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:46:23.946342 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bc863b-2530-4121-a570-5d45dd7d83ff admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:46:23.950810 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209dcf83-e88f-4cc6-b186-8116621f8d5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:46:23.951591 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209dcf83-e88f-4cc6-b186-8116621f8d5a admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:46:23.952514 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209dcf83-e88f-4cc6-b186-8116621f8d5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:46:23.952796 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209dcf83-e88f-4cc6-b186-8116621f8d5a admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:46:23.953526 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209dcf83-e88f-4cc6-b186-8116621f8d5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:46:23.955133 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209dcf83-e88f-4cc6-b186-8116621f8d5a admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:46:23.965555 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1377/2752] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:46:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:46:23.969593 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e5dde8-5b37-4daf-9713-2099e1fc5abb admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:46:23.979259 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e5dde8-5b37-4daf-9713-2099e1fc5abb admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:46:23.984134 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7035c428-98a3-4050-87a3-b068d8d677cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:46:23.984996 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7035c428-98a3-4050-87a3-b068d8d677cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:46:23.985278 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7035c428-98a3-4050-87a3-b068d8d677cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:46:23.985554 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7035c428-98a3-4050-87a3-b068d8d677cc admin admin] PATH_INFO: `/v3/projects/255d3f049fbf42de896ed43c0890cf19/users/a1f61a60f88542739cbc421988c32d57/roles/60af5088793e4379b88821a86552a344` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:46:24.000125 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7035c428-98a3-4050-87a3-b068d8d677cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=255d3f049fbf42de896ed43c0890cf19, user_id=a1f61a60f88542739cbc421988c32d57, role_id=60af5088793e4379b88821a86552a344)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:46:24.002720 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7035c428-98a3-4050-87a3-b068d8d677cc admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:46:24.131361 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1376/2753] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:46:23 2022] PUT /identity/v3/projects/255d3f049fbf42de896ed43c0890cf19/users/a1f61a60f88542739cbc421988c32d57/roles/60af5088793e4379b88821a86552a344 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 17 19:46:24.135271 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209dcf83-e88f-4cc6-b186-8116621f8d5a admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:46:24.145816 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209dcf83-e88f-4cc6-b186-8116621f8d5a admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:46:24.198394 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cac6b2-22d7-4afe-ae73-71074c59ff8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:46:24.199040 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cac6b2-22d7-4afe-ae73-71074c59ff8e admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:46:24.199316 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cac6b2-22d7-4afe-ae73-71074c59ff8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:46:24.199625 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cac6b2-22d7-4afe-ae73-71074c59ff8e admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:46:24.200332 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cac6b2-22d7-4afe-ae73-71074c59ff8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:46:24.202557 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cac6b2-22d7-4afe-ae73-71074c59ff8e admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:46:24.212035 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1378/2754] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:46:24 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:46:24.216115 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7035c428-98a3-4050-87a3-b068d8d677cc admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:46:24.226080 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7035c428-98a3-4050-87a3-b068d8d677cc admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:46:24.230579 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c2d4f9-297d-46f9-b560-6dd829d398a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:46:24.231436 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c2d4f9-297d-46f9-b560-6dd829d398a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:46:24.231743 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c2d4f9-297d-46f9-b560-6dd829d398a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:46:24.232011 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c2d4f9-297d-46f9-b560-6dd829d398a9 admin admin] PATH_INFO: `/v3/projects/255d3f049fbf42de896ed43c0890cf19/users/a1f61a60f88542739cbc421988c32d57/roles/60af5088793e4379b88821a86552a344` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:46:24.234741 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c2d4f9-297d-46f9-b560-6dd829d398a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=255d3f049fbf42de896ed43c0890cf19, user_id=a1f61a60f88542739cbc421988c32d57, role_id=60af5088793e4379b88821a86552a344)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:46:24.237379 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c2d4f9-297d-46f9-b560-6dd829d398a9 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:46:24.247013 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1377/2755] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:46:24 2022] PUT /identity/v3/projects/255d3f049fbf42de896ed43c0890cf19/users/a1f61a60f88542739cbc421988c32d57/roles/60af5088793e4379b88821a86552a344 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:46:24.257377 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27c47f6-c14b-4afa-9abe-4b82dddaa9a0 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:46:24.257604 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27c47f6-c14b-4afa-9abe-4b82dddaa9a0 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:46:24.257874 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27c47f6-c14b-4afa-9abe-4b82dddaa9a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:46:24.282719 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-f27c47f6-c14b-4afa-9abe-4b82dddaa9a0 None None] MFA Rules not processed for user `a1f61a60f88542739cbc421988c32d57`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:46:24.283947 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-f27c47f6-c14b-4afa-9abe-4b82dddaa9a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:46:24.349082 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1379/2756] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:46:24 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:46:24.361343 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c2d4f9-297d-46f9-b560-6dd829d398a9 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:46:24.362993 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-01c2d4f9-297d-46f9-b560-6dd829d398a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:46:24.416430 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c2d4f9-297d-46f9-b560-6dd829d398a9 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:46:24.420022 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771ba6f6-0564-4aa7-aad9-3f4996ed44d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd05409b38ae346458abfe41ebda1c59e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:46:24.421150 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771ba6f6-0564-4aa7-aad9-3f4996ed44d3 service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:46:24.421424 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771ba6f6-0564-4aa7-aad9-3f4996ed44d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:46:24.421692 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771ba6f6-0564-4aa7-aad9-3f4996ed44d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:46:24.430309 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771ba6f6-0564-4aa7-aad9-3f4996ed44d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:46:24.431745 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771ba6f6-0564-4aa7-aad9-3f4996ed44d3 service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:46:24.441406 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1378/2757] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:46:24 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:46:24.664769 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27c47f6-c14b-4afa-9abe-4b82dddaa9a0 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:46:24.675513 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27c47f6-c14b-4afa-9abe-4b82dddaa9a0 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:46:24.679256 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b316da88-e526-45f9-b9ca-ec32d8ff6ff0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd05409b38ae346458abfe41ebda1c59e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:46:24.679929 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b316da88-e526-45f9-b9ca-ec32d8ff6ff0 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:46:24.680203 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b316da88-e526-45f9-b9ca-ec32d8ff6ff0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:46:24.680473 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b316da88-e526-45f9-b9ca-ec32d8ff6ff0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:46:24.735286 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b316da88-e526-45f9-b9ca-ec32d8ff6ff0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:46:24.736772 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b316da88-e526-45f9-b9ca-ec32d8ff6ff0 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:46:24.745438 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1380/2758] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:46:24 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:46:43.653075 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771ba6f6-0564-4aa7-aad9-3f4996ed44d3 service cinder] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:46:43.712551 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771ba6f6-0564-4aa7-aad9-3f4996ed44d3 service cinder] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:46:43.717708 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628403a7-407f-4ce6-ae6a-b68d7ec7e021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:46:43.718559 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628403a7-407f-4ce6-ae6a-b68d7ec7e021 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:46:43.718835 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628403a7-407f-4ce6-ae6a-b68d7ec7e021 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:46:43.719108 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628403a7-407f-4ce6-ae6a-b68d7ec7e021 admin admin] PATH_INFO: `/v3/users/d92c85679c004cd7a758d82ba12e7d39` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:46:43.720301 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628403a7-407f-4ce6-ae6a-b68d7ec7e021 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d92c85679c004cd7a758d82ba12e7d39)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:46:43.722202 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628403a7-407f-4ce6-ae6a-b68d7ec7e021 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:46:43.894707 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-628403a7-407f-4ce6-ae6a-b68d7ec7e021 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd92c85679c004cd7a758d82ba12e7d39'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:46:43.901196 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-628403a7-407f-4ce6-ae6a-b68d7ec7e021 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd92c85679c004cd7a758d82ba12e7d39'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:46:43.913173 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-628403a7-407f-4ce6-ae6a-b68d7ec7e021 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd92c85679c004cd7a758d82ba12e7d39'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:46:43.913836 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-628403a7-407f-4ce6-ae6a-b68d7ec7e021 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd92c85679c004cd7a758d82ba12e7d39'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:46:43.914433 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-628403a7-407f-4ce6-ae6a-b68d7ec7e021 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd92c85679c004cd7a758d82ba12e7d39'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:46:43.993599 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1379/2759] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 17 19:46:43 2022] DELETE /identity/v3/users/d92c85679c004cd7a758d82ba12e7d39 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:46:43.998093 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aa95bd-8970-4433-b5e4-0b01c1c22f3c None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:46:43.998603 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aa95bd-8970-4433-b5e4-0b01c1c22f3c None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:46:43.998709 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aa95bd-8970-4433-b5e4-0b01c1c22f3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:46:44.015922 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-99aa95bd-8970-4433-b5e4-0b01c1c22f3c None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:46:44.017092 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-99aa95bd-8970-4433-b5e4-0b01c1c22f3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:46:44.073155 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1381/2760] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:46:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:46:44.083273 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628403a7-407f-4ce6-ae6a-b68d7ec7e021 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:46:44.084798 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-628403a7-407f-4ce6-ae6a-b68d7ec7e021 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:46:44.140842 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628403a7-407f-4ce6-ae6a-b68d7ec7e021 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:46:44.144660 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9071f8-aa36-4fa9-b159-7aa2acee4c60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:46:44.145415 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9071f8-aa36-4fa9-b159-7aa2acee4c60 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:46:44.145648 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9071f8-aa36-4fa9-b159-7aa2acee4c60 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:46:44.145923 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9071f8-aa36-4fa9-b159-7aa2acee4c60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:46:44.155376 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9071f8-aa36-4fa9-b159-7aa2acee4c60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:46:44.156962 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9071f8-aa36-4fa9-b159-7aa2acee4c60 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:46:44.166852 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1380/2761] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:46:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:46:44.578337 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aa95bd-8970-4433-b5e4-0b01c1c22f3c None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:46:44.579952 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-99aa95bd-8970-4433-b5e4-0b01c1c22f3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:46:44.591984 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aa95bd-8970-4433-b5e4-0b01c1c22f3c None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:46:44.596704 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d726f2d2-70ff-4605-8973-3d26e5148fb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:46:44.597560 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d726f2d2-70ff-4605-8973-3d26e5148fb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:46:44.597850 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d726f2d2-70ff-4605-8973-3d26e5148fb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:46:44.598293 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d726f2d2-70ff-4605-8973-3d26e5148fb2 admin admin] PATH_INFO: `/v3/projects/a0d9a07571374003a84ae4703d71d7a8` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:46:44.599371 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d726f2d2-70ff-4605-8973-3d26e5148fb2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0d9a07571374003a84ae4703d71d7a8)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:46:44.603266 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d726f2d2-70ff-4605-8973-3d26e5148fb2 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:46:44.754115 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d726f2d2-70ff-4605-8973-3d26e5148fb2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0d9a07571374003a84ae4703d71d7a8', 'request_id': 'req-d726f2d2-70ff-4605-8973-3d26e5148fb2'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:46:44.759257 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d726f2d2-70ff-4605-8973-3d26e5148fb2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0d9a07571374003a84ae4703d71d7a8', 'request_id': 'req-d726f2d2-70ff-4605-8973-3d26e5148fb2'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:46:44.811451 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d726f2d2-70ff-4605-8973-3d26e5148fb2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0d9a07571374003a84ae4703d71d7a8', 'request_id': 'req-d726f2d2-70ff-4605-8973-3d26e5148fb2'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:46:44.822855 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d726f2d2-70ff-4605-8973-3d26e5148fb2 admin admin] The token cache is being invalidate because project a0d9a07571374003a84ae4703d71d7a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 17 19:46:44.823140 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d726f2d2-70ff-4605-8973-3d26e5148fb2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:46:44.823776 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d726f2d2-70ff-4605-8973-3d26e5148fb2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:46:44.825346 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1382/2762] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 17 19:46:44 2022] DELETE /identity/v3/projects/a0d9a07571374003a84ae4703d71d7a8 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:46:44.829305 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9071f8-aa36-4fa9-b159-7aa2acee4c60 service neutron] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:46:44.838832 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9071f8-aa36-4fa9-b159-7aa2acee4c60 service neutron] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:46:44.892480 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e8d5f3-3abc-465c-b341-fe26c5106b94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:46:44.893340 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e8d5f3-3abc-465c-b341-fe26c5106b94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:46:44.893614 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e8d5f3-3abc-465c-b341-fe26c5106b94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:46:44.893883 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e8d5f3-3abc-465c-b341-fe26c5106b94 admin admin] PATH_INFO: `/v3/users/a1f61a60f88542739cbc421988c32d57` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:46:44.895036 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e8d5f3-3abc-465c-b341-fe26c5106b94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1f61a60f88542739cbc421988c32d57)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:46:44.896969 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e8d5f3-3abc-465c-b341-fe26c5106b94 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:46:45.069232 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-f9e8d5f3-3abc-465c-b341-fe26c5106b94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1f61a60f88542739cbc421988c32d57'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:46:45.075776 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-f9e8d5f3-3abc-465c-b341-fe26c5106b94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1f61a60f88542739cbc421988c32d57'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:46:45.086837 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-f9e8d5f3-3abc-465c-b341-fe26c5106b94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1f61a60f88542739cbc421988c32d57'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:46:45.087497 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-f9e8d5f3-3abc-465c-b341-fe26c5106b94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1f61a60f88542739cbc421988c32d57'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:46:45.088097 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-f9e8d5f3-3abc-465c-b341-fe26c5106b94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1f61a60f88542739cbc421988c32d57'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:46:45.176542 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1381/2763] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 17 19:46:44 2022] DELETE /identity/v3/users/a1f61a60f88542739cbc421988c32d57 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:46:45.538311 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d726f2d2-70ff-4605-8973-3d26e5148fb2 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:46:45.539886 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d726f2d2-70ff-4605-8973-3d26e5148fb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:46:45.554950 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d726f2d2-70ff-4605-8973-3d26e5148fb2 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:46:45.611313 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad1318f-7fe5-4d0b-a446-15edb48906b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:46:45.612206 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad1318f-7fe5-4d0b-a446-15edb48906b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:46:45.612481 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad1318f-7fe5-4d0b-a446-15edb48906b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:46:45.612783 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad1318f-7fe5-4d0b-a446-15edb48906b0 admin admin] PATH_INFO: `/v3/projects/255d3f049fbf42de896ed43c0890cf19` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:46:45.614076 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad1318f-7fe5-4d0b-a446-15edb48906b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=255d3f049fbf42de896ed43c0890cf19)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:46:45.616036 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad1318f-7fe5-4d0b-a446-15edb48906b0 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:46:45.734762 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-8ad1318f-7fe5-4d0b-a446-15edb48906b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '255d3f049fbf42de896ed43c0890cf19', 'request_id': 'req-8ad1318f-7fe5-4d0b-a446-15edb48906b0'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:46:45.740112 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-8ad1318f-7fe5-4d0b-a446-15edb48906b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '255d3f049fbf42de896ed43c0890cf19', 'request_id': 'req-8ad1318f-7fe5-4d0b-a446-15edb48906b0'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:46:45.805177 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-8ad1318f-7fe5-4d0b-a446-15edb48906b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '255d3f049fbf42de896ed43c0890cf19', 'request_id': 'req-8ad1318f-7fe5-4d0b-a446-15edb48906b0'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:46:45.816855 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-8ad1318f-7fe5-4d0b-a446-15edb48906b0 admin admin] The token cache is being invalidate because project 255d3f049fbf42de896ed43c0890cf19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 17 19:46:45.817139 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-8ad1318f-7fe5-4d0b-a446-15edb48906b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:46:45.817757 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-8ad1318f-7fe5-4d0b-a446-15edb48906b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:46:45.819337 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1383/2764] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 17 19:46:45 2022] DELETE /identity/v3/projects/255d3f049fbf42de896ed43c0890cf19 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:46:45.824348 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edbd752-9e02-40c0-8928-7a60f3609ff4 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:46:45.824628 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edbd752-9e02-40c0-8928-7a60f3609ff4 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:46:45.824898 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edbd752-9e02-40c0-8928-7a60f3609ff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:46:45.844718 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-5edbd752-9e02-40c0-8928-7a60f3609ff4 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:46:45.845897 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-5edbd752-9e02-40c0-8928-7a60f3609ff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:46:45.904069 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1382/2765] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:46:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:46:45.912557 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f61fe2b-8bd9-46b8-a256-d86bf0d9b829 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:46:45.912891 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f61fe2b-8bd9-46b8-a256-d86bf0d9b829 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:46:45.913236 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f61fe2b-8bd9-46b8-a256-d86bf0d9b829 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:46:45.933251 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-2f61fe2b-8bd9-46b8-a256-d86bf0d9b829 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:46:45.934515 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-2f61fe2b-8bd9-46b8-a256-d86bf0d9b829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:46:45.943253 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1384/2766] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:46:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:46:45.947606 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edbd752-9e02-40c0-8928-7a60f3609ff4 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:46:45.957752 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edbd752-9e02-40c0-8928-7a60f3609ff4 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:46:45.962315 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0271e9-b9a6-48a8-aff7-fa5ce2e288b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:46:45.962952 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0271e9-b9a6-48a8-aff7-fa5ce2e288b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:46:45.963230 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0271e9-b9a6-48a8-aff7-fa5ce2e288b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:46:45.963534 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0271e9-b9a6-48a8-aff7-fa5ce2e288b0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:46:45.964251 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0271e9-b9a6-48a8-aff7-fa5ce2e288b0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:46:45.964551 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0271e9-b9a6-48a8-aff7-fa5ce2e288b0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:46:45.966157 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0271e9-b9a6-48a8-aff7-fa5ce2e288b0 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:46:45.977173 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1383/2767] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 17 19:46:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:46:45.982127 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f61fe2b-8bd9-46b8-a256-d86bf0d9b829 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:46:45.991334 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f61fe2b-8bd9-46b8-a256-d86bf0d9b829 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:46:45.995988 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651055ed-a1f6-44fa-8b9a-373d8f80c8a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:46:45.996619 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651055ed-a1f6-44fa-8b9a-373d8f80c8a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:46:45.996896 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651055ed-a1f6-44fa-8b9a-373d8f80c8a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:46:45.997171 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651055ed-a1f6-44fa-8b9a-373d8f80c8a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:46:45.998455 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651055ed-a1f6-44fa-8b9a-373d8f80c8a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:46:46.000386 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651055ed-a1f6-44fa-8b9a-373d8f80c8a2 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:46:46.069108 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1385/2768] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 17 19:46:45 2022] POST /identity/v3/projects => generated 373 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:46:46.072878 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0271e9-b9a6-48a8-aff7-fa5ce2e288b0 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:46:46.082580 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0271e9-b9a6-48a8-aff7-fa5ce2e288b0 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:46:46.136287 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6387e673-0a23-45d7-8b82-eb2646337abb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:46:46.136937 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6387e673-0a23-45d7-8b82-eb2646337abb admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:46:46.137246 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6387e673-0a23-45d7-8b82-eb2646337abb admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:46:46.137523 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6387e673-0a23-45d7-8b82-eb2646337abb admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:46:46.138217 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6387e673-0a23-45d7-8b82-eb2646337abb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:46:46.140152 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6387e673-0a23-45d7-8b82-eb2646337abb admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:46:46.234202 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1384/2769] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 17 19:46:46 2022] POST /identity/v3/users => generated 287 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:46:46.238520 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651055ed-a1f6-44fa-8b9a-373d8f80c8a2 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:46:46.249264 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651055ed-a1f6-44fa-8b9a-373d8f80c8a2 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:46:46.253875 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ff80d4-4777-4586-976c-d67da698f9ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:46:46.254596 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ff80d4-4777-4586-976c-d67da698f9ce admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:46:46.254873 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ff80d4-4777-4586-976c-d67da698f9ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:46:46.255169 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ff80d4-4777-4586-976c-d67da698f9ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:46:46.255991 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ff80d4-4777-4586-976c-d67da698f9ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:46:46.257575 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ff80d4-4777-4586-976c-d67da698f9ce admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:46:46.267831 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1386/2770] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:46:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:46:46.271586 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6387e673-0a23-45d7-8b82-eb2646337abb admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:46:46.281395 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6387e673-0a23-45d7-8b82-eb2646337abb admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:46:46.285997 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ad94a3-7e5d-42e1-889c-d38105885ee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:46:46.286636 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ad94a3-7e5d-42e1-889c-d38105885ee1 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:46:46.286916 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ad94a3-7e5d-42e1-889c-d38105885ee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:46:46.287185 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ad94a3-7e5d-42e1-889c-d38105885ee1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:46:46.287930 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ad94a3-7e5d-42e1-889c-d38105885ee1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:46:46.289500 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ad94a3-7e5d-42e1-889c-d38105885ee1 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:46:46.299596 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1385/2771] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:46:46 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:46:46.303331 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ff80d4-4777-4586-976c-d67da698f9ce admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:46:46.313462 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ff80d4-4777-4586-976c-d67da698f9ce admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:46:46.317969 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b1a635-b5f8-4e3e-9365-234b47057833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:46:46.318823 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b1a635-b5f8-4e3e-9365-234b47057833 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:46:46.319099 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b1a635-b5f8-4e3e-9365-234b47057833 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:46:46.319420 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b1a635-b5f8-4e3e-9365-234b47057833 admin admin] PATH_INFO: `/v3/projects/124b73cc85da446b92a12a1609400fa9/users/cd30531696ee46838259b38c39369c27/roles/114e6a16e63447d1924707ace5aace25` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:46:46.333150 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b1a635-b5f8-4e3e-9365-234b47057833 admin admin] RBAC: Authorizing `identity:create_grant(project_id=124b73cc85da446b92a12a1609400fa9, user_id=cd30531696ee46838259b38c39369c27, role_id=114e6a16e63447d1924707ace5aace25)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:46:46.335797 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b1a635-b5f8-4e3e-9365-234b47057833 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:46:46.404366 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1387/2772] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:46:46 2022] PUT /identity/v3/projects/124b73cc85da446b92a12a1609400fa9/users/cd30531696ee46838259b38c39369c27/roles/114e6a16e63447d1924707ace5aace25 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:46:46.414421 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2146c39a-66c1-4fa6-9bf4-4bdd042b85d8 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:46:46.414711 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2146c39a-66c1-4fa6-9bf4-4bdd042b85d8 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:46:46.414989 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2146c39a-66c1-4fa6-9bf4-4bdd042b85d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:46:46.440954 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-2146c39a-66c1-4fa6-9bf4-4bdd042b85d8 None None] MFA Rules not processed for user `cd30531696ee46838259b38c39369c27`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:46:46.443050 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-2146c39a-66c1-4fa6-9bf4-4bdd042b85d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:46:46.503726 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1386/2773] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:46:46 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:46:46.508386 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b1a635-b5f8-4e3e-9365-234b47057833 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:46:46.571279 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b1a635-b5f8-4e3e-9365-234b47057833 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:46:46.576585 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a11df5-0bcc-4325-bb14-aa4ec93c0f77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:46:46.577226 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a11df5-0bcc-4325-bb14-aa4ec93c0f77 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:46:46.577502 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a11df5-0bcc-4325-bb14-aa4ec93c0f77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:46:46.577773 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a11df5-0bcc-4325-bb14-aa4ec93c0f77 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:46:46.578458 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a11df5-0bcc-4325-bb14-aa4ec93c0f77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:46:46.580415 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a11df5-0bcc-4325-bb14-aa4ec93c0f77 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:46:46.720537 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1388/2774] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 17 19:46:46 2022] POST /identity/v3/projects => generated 377 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:46:46.724446 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2146c39a-66c1-4fa6-9bf4-4bdd042b85d8 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:46:46.735619 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2146c39a-66c1-4fa6-9bf4-4bdd042b85d8 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:46:46.788809 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7398a8d-5798-489d-b950-53ddbe98e338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:46:46.789467 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7398a8d-5798-489d-b950-53ddbe98e338 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:46:46.789743 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7398a8d-5798-489d-b950-53ddbe98e338 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:46:46.790012 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7398a8d-5798-489d-b950-53ddbe98e338 admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:46:46.790679 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7398a8d-5798-489d-b950-53ddbe98e338 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:46:46.792632 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7398a8d-5798-489d-b950-53ddbe98e338 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:46:46.897255 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1387/2775] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 17 19:46:46 2022] POST /identity/v3/users => generated 295 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:46:46.901047 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a11df5-0bcc-4325-bb14-aa4ec93c0f77 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:46:46.911263 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a11df5-0bcc-4325-bb14-aa4ec93c0f77 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:46:46.915901 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a700b6-fd86-48be-83e1-b32676f50856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:46:46.916547 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a700b6-fd86-48be-83e1-b32676f50856 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:46:46.916832 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a700b6-fd86-48be-83e1-b32676f50856 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:46:46.917106 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a700b6-fd86-48be-83e1-b32676f50856 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:46:46.917814 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a700b6-fd86-48be-83e1-b32676f50856 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:46:46.919971 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a700b6-fd86-48be-83e1-b32676f50856 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:46:46.930217 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1389/2776] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:46:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:46:46.936314 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7398a8d-5798-489d-b950-53ddbe98e338 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:46:46.945242 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7398a8d-5798-489d-b950-53ddbe98e338 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:46:46.950068 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1417198-d43c-4d38-b901-3119bdfcecd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:46:46.950932 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1417198-d43c-4d38-b901-3119bdfcecd3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:46:46.951205 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1417198-d43c-4d38-b901-3119bdfcecd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:46:46.951470 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1417198-d43c-4d38-b901-3119bdfcecd3 admin admin] PATH_INFO: `/v3/projects/d3384abdf5f042eda813580bf84b4124/users/982bafc981014c129f00f051b7dd51c4/roles/60af5088793e4379b88821a86552a344` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:46:46.965992 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1417198-d43c-4d38-b901-3119bdfcecd3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3384abdf5f042eda813580bf84b4124, user_id=982bafc981014c129f00f051b7dd51c4, role_id=60af5088793e4379b88821a86552a344)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:46:46.968667 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1417198-d43c-4d38-b901-3119bdfcecd3 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:46:47.038852 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1388/2777] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:46:46 2022] PUT /identity/v3/projects/d3384abdf5f042eda813580bf84b4124/users/982bafc981014c129f00f051b7dd51c4/roles/60af5088793e4379b88821a86552a344 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 17 19:46:47.042777 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a700b6-fd86-48be-83e1-b32676f50856 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:46:47.052477 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a700b6-fd86-48be-83e1-b32676f50856 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:46:47.105062 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85c5176-1d89-495e-905d-8465daae4411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:46:47.105706 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85c5176-1d89-495e-905d-8465daae4411 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:46:47.105983 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85c5176-1d89-495e-905d-8465daae4411 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:46:47.106254 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85c5176-1d89-495e-905d-8465daae4411 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:46:47.106984 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85c5176-1d89-495e-905d-8465daae4411 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:46:47.108857 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85c5176-1d89-495e-905d-8465daae4411 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:46:47.118071 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1390/2778] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:46:47 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:46:47.123106 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1417198-d43c-4d38-b901-3119bdfcecd3 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:46:47.133693 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1417198-d43c-4d38-b901-3119bdfcecd3 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:46:47.139067 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32546ac-5149-4213-8212-9da42654aa1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:46:47.140054 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32546ac-5149-4213-8212-9da42654aa1b admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:46:47.140380 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32546ac-5149-4213-8212-9da42654aa1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:46:47.140651 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32546ac-5149-4213-8212-9da42654aa1b admin admin] PATH_INFO: `/v3/projects/d3384abdf5f042eda813580bf84b4124/users/982bafc981014c129f00f051b7dd51c4/roles/60af5088793e4379b88821a86552a344` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:46:47.142707 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32546ac-5149-4213-8212-9da42654aa1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3384abdf5f042eda813580bf84b4124, user_id=982bafc981014c129f00f051b7dd51c4, role_id=60af5088793e4379b88821a86552a344)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:46:47.145575 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32546ac-5149-4213-8212-9da42654aa1b admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:46:47.155842 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1389/2779] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:46:47 2022] PUT /identity/v3/projects/d3384abdf5f042eda813580bf84b4124/users/982bafc981014c129f00f051b7dd51c4/roles/60af5088793e4379b88821a86552a344 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:46:47.216336 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437b7350-ffd7-4157-a49c-9a03d560c8d3 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:46:47.216616 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437b7350-ffd7-4157-a49c-9a03d560c8d3 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:46:47.216891 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437b7350-ffd7-4157-a49c-9a03d560c8d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:46:47.243501 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-437b7350-ffd7-4157-a49c-9a03d560c8d3 None None] MFA Rules not processed for user `982bafc981014c129f00f051b7dd51c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:46:47.243501 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-437b7350-ffd7-4157-a49c-9a03d560c8d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:46:47.311260 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1391/2780] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:46:47 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:46:47.322225 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32546ac-5149-4213-8212-9da42654aa1b admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:46:47.323937 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-c32546ac-5149-4213-8212-9da42654aa1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:46:47.379254 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32546ac-5149-4213-8212-9da42654aa1b admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:46:47.382826 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c24d27-7a54-4413-b67a-9159b9631321 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd05409b38ae346458abfe41ebda1c59e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:46:47.383448 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c24d27-7a54-4413-b67a-9159b9631321 service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:46:47.383753 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c24d27-7a54-4413-b67a-9159b9631321 service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:46:47.384024 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c24d27-7a54-4413-b67a-9159b9631321 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:46:47.436934 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c24d27-7a54-4413-b67a-9159b9631321 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:46:47.438295 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c24d27-7a54-4413-b67a-9159b9631321 service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:46:47.447083 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1390/2781] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:46:47 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:46:53.908694 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437b7350-ffd7-4157-a49c-9a03d560c8d3 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:46:53.910091 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-437b7350-ffd7-4157-a49c-9a03d560c8d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:46:53.921728 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437b7350-ffd7-4157-a49c-9a03d560c8d3 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:46:53.925237 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8366c4b4-0e40-429f-a756-3650cc672cfd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd05409b38ae346458abfe41ebda1c59e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:46:53.925860 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8366c4b4-0e40-429f-a756-3650cc672cfd service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:46:53.926136 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8366c4b4-0e40-429f-a756-3650cc672cfd service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:46:53.926407 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8366c4b4-0e40-429f-a756-3650cc672cfd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:46:53.935722 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8366c4b4-0e40-429f-a756-3650cc672cfd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:46:53.937181 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8366c4b4-0e40-429f-a756-3650cc672cfd service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:46:53.946674 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1392/2782] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:46:53 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:47:21.195031 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c24d27-7a54-4413-b67a-9159b9631321 service cinder] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:47:21.254680 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c24d27-7a54-4413-b67a-9159b9631321 service cinder] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:47:21.259062 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b160718-07e1-4195-b159-1da509b82ba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:47:21.260008 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b160718-07e1-4195-b159-1da509b82ba2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:47:21.260289 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b160718-07e1-4195-b159-1da509b82ba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:47:21.260562 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b160718-07e1-4195-b159-1da509b82ba2 admin admin] PATH_INFO: `/v3/users/cd30531696ee46838259b38c39369c27` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:47:21.261720 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b160718-07e1-4195-b159-1da509b82ba2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd30531696ee46838259b38c39369c27)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:47:21.263649 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b160718-07e1-4195-b159-1da509b82ba2 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:47:21.434195 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-4b160718-07e1-4195-b159-1da509b82ba2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd30531696ee46838259b38c39369c27'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:47:21.440818 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-4b160718-07e1-4195-b159-1da509b82ba2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd30531696ee46838259b38c39369c27'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:47:21.452015 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-4b160718-07e1-4195-b159-1da509b82ba2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd30531696ee46838259b38c39369c27'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:47:21.452672 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-4b160718-07e1-4195-b159-1da509b82ba2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd30531696ee46838259b38c39369c27'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:47:21.453275 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-4b160718-07e1-4195-b159-1da509b82ba2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd30531696ee46838259b38c39369c27'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:47:21.547405 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1391/2783] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 17 19:47:21 2022] DELETE /identity/v3/users/cd30531696ee46838259b38c39369c27 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:47:21.552037 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e48f4b-d107-4bee-aa50-9650b79ac2c3 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:47:21.552571 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e48f4b-d107-4bee-aa50-9650b79ac2c3 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:47:21.552571 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e48f4b-d107-4bee-aa50-9650b79ac2c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:47:21.570016 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-08e48f4b-d107-4bee-aa50-9650b79ac2c3 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:47:21.578773 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-08e48f4b-d107-4bee-aa50-9650b79ac2c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:47:21.628503 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1393/2784] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:47:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:47:21.638099 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b160718-07e1-4195-b159-1da509b82ba2 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:47:21.639612 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-4b160718-07e1-4195-b159-1da509b82ba2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:47:21.695145 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b160718-07e1-4195-b159-1da509b82ba2 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:47:21.698776 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4273bb0-c8be-4276-985a-3e21e54c97e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:47:21.699404 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4273bb0-c8be-4276-985a-3e21e54c97e3 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:47:21.699707 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4273bb0-c8be-4276-985a-3e21e54c97e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:47:21.699977 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4273bb0-c8be-4276-985a-3e21e54c97e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:47:21.709413 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4273bb0-c8be-4276-985a-3e21e54c97e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:47:21.710824 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4273bb0-c8be-4276-985a-3e21e54c97e3 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:47:21.720044 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1392/2785] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:47:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:47:22.199089 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e48f4b-d107-4bee-aa50-9650b79ac2c3 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:47:22.200615 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-08e48f4b-d107-4bee-aa50-9650b79ac2c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:47:22.212795 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e48f4b-d107-4bee-aa50-9650b79ac2c3 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:47:22.217578 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a441a4a-8000-4b18-8abd-0d72b6ca262a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:47:22.218488 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a441a4a-8000-4b18-8abd-0d72b6ca262a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:47:22.224229 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a441a4a-8000-4b18-8abd-0d72b6ca262a admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:47:22.224229 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a441a4a-8000-4b18-8abd-0d72b6ca262a admin admin] PATH_INFO: `/v3/projects/124b73cc85da446b92a12a1609400fa9` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:47:22.224229 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a441a4a-8000-4b18-8abd-0d72b6ca262a admin admin] RBAC: Authorizing `identity:delete_project(project_id=124b73cc85da446b92a12a1609400fa9)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:47:22.224229 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a441a4a-8000-4b18-8abd-0d72b6ca262a admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:47:22.335019 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0a441a4a-8000-4b18-8abd-0d72b6ca262a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '124b73cc85da446b92a12a1609400fa9', 'request_id': 'req-0a441a4a-8000-4b18-8abd-0d72b6ca262a'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:47:22.340405 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0a441a4a-8000-4b18-8abd-0d72b6ca262a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '124b73cc85da446b92a12a1609400fa9', 'request_id': 'req-0a441a4a-8000-4b18-8abd-0d72b6ca262a'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:47:22.421917 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0a441a4a-8000-4b18-8abd-0d72b6ca262a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '124b73cc85da446b92a12a1609400fa9', 'request_id': 'req-0a441a4a-8000-4b18-8abd-0d72b6ca262a'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:47:22.433510 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0a441a4a-8000-4b18-8abd-0d72b6ca262a admin admin] The token cache is being invalidate because project 124b73cc85da446b92a12a1609400fa9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 17 19:47:22.433798 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0a441a4a-8000-4b18-8abd-0d72b6ca262a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:47:22.434395 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0a441a4a-8000-4b18-8abd-0d72b6ca262a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:47:22.435958 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1394/2786] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 17 19:47:22 2022] DELETE /identity/v3/projects/124b73cc85da446b92a12a1609400fa9 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:47:22.441529 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4273bb0-c8be-4276-985a-3e21e54c97e3 service neutron] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:47:22.450031 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4273bb0-c8be-4276-985a-3e21e54c97e3 service neutron] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:47:22.504622 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e4bb0d-d45d-47b6-b22e-6120afe80f38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:47:22.505480 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e4bb0d-d45d-47b6-b22e-6120afe80f38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:47:22.505758 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e4bb0d-d45d-47b6-b22e-6120afe80f38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:47:22.506028 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e4bb0d-d45d-47b6-b22e-6120afe80f38 admin admin] PATH_INFO: `/v3/users/982bafc981014c129f00f051b7dd51c4` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:47:22.507194 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e4bb0d-d45d-47b6-b22e-6120afe80f38 admin admin] RBAC: Authorizing `identity:delete_user(user_id=982bafc981014c129f00f051b7dd51c4)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:47:22.509145 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e4bb0d-d45d-47b6-b22e-6120afe80f38 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:47:22.681110 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-57e4bb0d-d45d-47b6-b22e-6120afe80f38 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '982bafc981014c129f00f051b7dd51c4'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:47:22.687836 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-57e4bb0d-d45d-47b6-b22e-6120afe80f38 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '982bafc981014c129f00f051b7dd51c4'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:47:22.699301 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-57e4bb0d-d45d-47b6-b22e-6120afe80f38 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '982bafc981014c129f00f051b7dd51c4'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:47:22.700000 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-57e4bb0d-d45d-47b6-b22e-6120afe80f38 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '982bafc981014c129f00f051b7dd51c4'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:47:22.700733 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-57e4bb0d-d45d-47b6-b22e-6120afe80f38 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '982bafc981014c129f00f051b7dd51c4'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:47:22.798563 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1393/2787] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 17 19:47:22 2022] DELETE /identity/v3/users/982bafc981014c129f00f051b7dd51c4 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:47:23.234838 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a441a4a-8000-4b18-8abd-0d72b6ca262a admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:47:23.236372 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-0a441a4a-8000-4b18-8abd-0d72b6ca262a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:47:23.249459 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a441a4a-8000-4b18-8abd-0d72b6ca262a admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:47:23.305270 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a0a815-b5e1-4951-b016-45c6b96fcfff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:47:23.306151 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a0a815-b5e1-4951-b016-45c6b96fcfff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:47:23.306426 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a0a815-b5e1-4951-b016-45c6b96fcfff admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:47:23.306696 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a0a815-b5e1-4951-b016-45c6b96fcfff admin admin] PATH_INFO: `/v3/projects/d3384abdf5f042eda813580bf84b4124` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:47:23.307924 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a0a815-b5e1-4951-b016-45c6b96fcfff admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3384abdf5f042eda813580bf84b4124)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:47:23.309852 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a0a815-b5e1-4951-b016-45c6b96fcfff admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:47:23.438329 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-33a0a815-b5e1-4951-b016-45c6b96fcfff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3384abdf5f042eda813580bf84b4124', 'request_id': 'req-33a0a815-b5e1-4951-b016-45c6b96fcfff'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:47:23.444107 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-33a0a815-b5e1-4951-b016-45c6b96fcfff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3384abdf5f042eda813580bf84b4124', 'request_id': 'req-33a0a815-b5e1-4951-b016-45c6b96fcfff'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:47:23.530265 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-33a0a815-b5e1-4951-b016-45c6b96fcfff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3384abdf5f042eda813580bf84b4124', 'request_id': 'req-33a0a815-b5e1-4951-b016-45c6b96fcfff'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:47:23.541966 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-33a0a815-b5e1-4951-b016-45c6b96fcfff admin admin] The token cache is being invalidate because project d3384abdf5f042eda813580bf84b4124 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 17 19:47:23.542250 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-33a0a815-b5e1-4951-b016-45c6b96fcfff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:47:23.542868 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-33a0a815-b5e1-4951-b016-45c6b96fcfff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:47:23.544457 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1395/2788] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 17 19:47:23 2022] DELETE /identity/v3/projects/d3384abdf5f042eda813580bf84b4124 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:47:23.549274 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7d37ab-d112-41e2-b611-9dc2966146bd None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:47:23.549806 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7d37ab-d112-41e2-b611-9dc2966146bd None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:47:23.549922 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7d37ab-d112-41e2-b611-9dc2966146bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:47:23.567656 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-fb7d37ab-d112-41e2-b611-9dc2966146bd None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:47:23.568907 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-fb7d37ab-d112-41e2-b611-9dc2966146bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:47:23.626770 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1394/2789] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:47:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:47:23.634864 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c184906b-75f4-41fe-9e3b-6aa0bec4e513 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:47:23.635178 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c184906b-75f4-41fe-9e3b-6aa0bec4e513 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:47:23.635452 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c184906b-75f4-41fe-9e3b-6aa0bec4e513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:47:23.653876 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-c184906b-75f4-41fe-9e3b-6aa0bec4e513 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:47:23.655137 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c184906b-75f4-41fe-9e3b-6aa0bec4e513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:47:23.663140 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1396/2790] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:47:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:47:23.671034 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7d37ab-d112-41e2-b611-9dc2966146bd None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:47:23.680884 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7d37ab-d112-41e2-b611-9dc2966146bd None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:47:23.685437 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d943215-e307-4bbb-8f89-301b96418793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:47:23.686069 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d943215-e307-4bbb-8f89-301b96418793 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:47:23.686344 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d943215-e307-4bbb-8f89-301b96418793 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:47:23.686610 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d943215-e307-4bbb-8f89-301b96418793 admin admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:47:23.687325 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d943215-e307-4bbb-8f89-301b96418793 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:47:23.687654 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d943215-e307-4bbb-8f89-301b96418793 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:47:23.689344 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d943215-e307-4bbb-8f89-301b96418793 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:47:23.701306 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1395/2791] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 17 19:47:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:47:23.709134 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c184906b-75f4-41fe-9e3b-6aa0bec4e513 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:47:23.718178 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c184906b-75f4-41fe-9e3b-6aa0bec4e513 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:47:23.723009 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02486c1-a383-4335-b996-33486a5e8b3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:47:23.723743 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02486c1-a383-4335-b996-33486a5e8b3c admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:47:23.724047 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02486c1-a383-4335-b996-33486a5e8b3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:47:23.724314 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02486c1-a383-4335-b996-33486a5e8b3c admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:47:23.724984 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02486c1-a383-4335-b996-33486a5e8b3c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:47:23.726918 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02486c1-a383-4335-b996-33486a5e8b3c admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:47:23.845320 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1397/2792] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 17 19:47:23 2022] POST /identity/v3/projects => generated 387 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:47:23.850045 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d943215-e307-4bbb-8f89-301b96418793 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:47:23.860056 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d943215-e307-4bbb-8f89-301b96418793 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:47:23.913470 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de1b187-1909-44a2-9a26-6917fc1c5c98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:47:23.914126 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de1b187-1909-44a2-9a26-6917fc1c5c98 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:47:23.914402 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de1b187-1909-44a2-9a26-6917fc1c5c98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:47:23.914677 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de1b187-1909-44a2-9a26-6917fc1c5c98 admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:47:23.915338 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de1b187-1909-44a2-9a26-6917fc1c5c98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:47:23.917265 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de1b187-1909-44a2-9a26-6917fc1c5c98 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:47:23.987138 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1396/2793] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 17 19:47:23 2022] POST /identity/v3/users => generated 294 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 17 19:47:23.991418 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02486c1-a383-4335-b996-33486a5e8b3c admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:47:24.001764 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02486c1-a383-4335-b996-33486a5e8b3c admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:47:24.006365 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e68445-3158-4005-afe5-019d5b901108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:47:24.007006 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e68445-3158-4005-afe5-019d5b901108 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:47:24.007284 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e68445-3158-4005-afe5-019d5b901108 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:47:24.007587 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e68445-3158-4005-afe5-019d5b901108 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:47:24.008297 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e68445-3158-4005-afe5-019d5b901108 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:47:24.009891 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e68445-3158-4005-afe5-019d5b901108 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:47:24.020349 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1398/2794] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:47:23 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:47:24.024310 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de1b187-1909-44a2-9a26-6917fc1c5c98 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:47:24.034774 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de1b187-1909-44a2-9a26-6917fc1c5c98 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:47:24.039454 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554c8008-f5db-4a33-82a1-df3ccbfa444c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:47:24.040124 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554c8008-f5db-4a33-82a1-df3ccbfa444c admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:47:24.040402 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554c8008-f5db-4a33-82a1-df3ccbfa444c admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:47:24.040676 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554c8008-f5db-4a33-82a1-df3ccbfa444c admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:47:24.041388 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554c8008-f5db-4a33-82a1-df3ccbfa444c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:47:24.042965 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554c8008-f5db-4a33-82a1-df3ccbfa444c admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:47:24.052726 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1397/2795] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:47:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:47:24.056683 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e68445-3158-4005-afe5-019d5b901108 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:47:24.066917 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e68445-3158-4005-afe5-019d5b901108 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:47:24.071660 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4fae12-1c5d-45fa-b6e8-ea1b0d2f9409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:47:24.072517 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4fae12-1c5d-45fa-b6e8-ea1b0d2f9409 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:47:24.072790 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4fae12-1c5d-45fa-b6e8-ea1b0d2f9409 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:47:24.073058 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4fae12-1c5d-45fa-b6e8-ea1b0d2f9409 admin admin] PATH_INFO: `/v3/projects/ecd4b95cf99340639585f8ea27b69b00/users/c56ed194fc3f4b6886fb6d36dcf09e43/roles/114e6a16e63447d1924707ace5aace25` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:47:24.087833 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4fae12-1c5d-45fa-b6e8-ea1b0d2f9409 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ecd4b95cf99340639585f8ea27b69b00, user_id=c56ed194fc3f4b6886fb6d36dcf09e43, role_id=114e6a16e63447d1924707ace5aace25)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:47:24.090476 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4fae12-1c5d-45fa-b6e8-ea1b0d2f9409 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:47:24.204107 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1399/2796] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:47:24 2022] PUT /identity/v3/projects/ecd4b95cf99340639585f8ea27b69b00/users/c56ed194fc3f4b6886fb6d36dcf09e43/roles/114e6a16e63447d1924707ace5aace25 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:47:24.215719 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7ae91e-0e63-4794-ac7f-89c476dd5187 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:47:24.216011 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7ae91e-0e63-4794-ac7f-89c476dd5187 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:47:24.216285 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7ae91e-0e63-4794-ac7f-89c476dd5187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:47:24.241949 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-4e7ae91e-0e63-4794-ac7f-89c476dd5187 None None] MFA Rules not processed for user `c56ed194fc3f4b6886fb6d36dcf09e43`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:47:24.243196 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-4e7ae91e-0e63-4794-ac7f-89c476dd5187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:47:24.304101 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1398/2797] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:47:24 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:47:24.308578 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4fae12-1c5d-45fa-b6e8-ea1b0d2f9409 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:47:24.367454 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4fae12-1c5d-45fa-b6e8-ea1b0d2f9409 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:47:24.372079 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399580ed-274a-49c2-8a7e-734b37d7ff48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:47:24.372729 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399580ed-274a-49c2-8a7e-734b37d7ff48 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:47:24.373009 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399580ed-274a-49c2-8a7e-734b37d7ff48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:47:24.373282 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399580ed-274a-49c2-8a7e-734b37d7ff48 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:47:24.373999 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399580ed-274a-49c2-8a7e-734b37d7ff48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:47:24.375950 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399580ed-274a-49c2-8a7e-734b37d7ff48 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:47:24.480825 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1400/2798] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 17 19:47:24 2022] POST /identity/v3/projects => generated 387 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:47:24.484405 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7ae91e-0e63-4794-ac7f-89c476dd5187 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:47:24.495268 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7ae91e-0e63-4794-ac7f-89c476dd5187 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:47:24.548117 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2325d63b-d270-4274-8e4d-e7b207047e4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:47:24.548771 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2325d63b-d270-4274-8e4d-e7b207047e4c admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:47:24.549552 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2325d63b-d270-4274-8e4d-e7b207047e4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:47:24.550105 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2325d63b-d270-4274-8e4d-e7b207047e4c admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:47:24.550493 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2325d63b-d270-4274-8e4d-e7b207047e4c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:47:24.552452 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2325d63b-d270-4274-8e4d-e7b207047e4c admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:47:24.685131 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1399/2799] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 17 19:47:24 2022] POST /identity/v3/users => generated 300 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:47:24.688969 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399580ed-274a-49c2-8a7e-734b37d7ff48 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:47:24.699469 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399580ed-274a-49c2-8a7e-734b37d7ff48 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:47:24.703962 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c43a871-340c-4dfe-b032-a0e971ae2026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:47:24.704603 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c43a871-340c-4dfe-b032-a0e971ae2026 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:47:24.704881 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c43a871-340c-4dfe-b032-a0e971ae2026 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:47:24.705153 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c43a871-340c-4dfe-b032-a0e971ae2026 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:47:24.705914 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c43a871-340c-4dfe-b032-a0e971ae2026 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:47:24.707507 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c43a871-340c-4dfe-b032-a0e971ae2026 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:47:24.717736 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1401/2800] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:47:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:47:24.725606 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2325d63b-d270-4274-8e4d-e7b207047e4c admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:47:24.735623 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2325d63b-d270-4274-8e4d-e7b207047e4c admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:47:24.740667 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2a267f-1ac9-472e-ac5b-ec7a416dc063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:47:24.741518 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2a267f-1ac9-472e-ac5b-ec7a416dc063 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:47:24.741793 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2a267f-1ac9-472e-ac5b-ec7a416dc063 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:47:24.742063 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2a267f-1ac9-472e-ac5b-ec7a416dc063 admin admin] PATH_INFO: `/v3/projects/eceb00f160194615adc439f1e2bb0e22/users/dfbcad775058475e8a295c3244f2d01b/roles/60af5088793e4379b88821a86552a344` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:47:24.756750 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2a267f-1ac9-472e-ac5b-ec7a416dc063 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eceb00f160194615adc439f1e2bb0e22, user_id=dfbcad775058475e8a295c3244f2d01b, role_id=60af5088793e4379b88821a86552a344)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:47:24.759362 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2a267f-1ac9-472e-ac5b-ec7a416dc063 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:47:24.851041 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1400/2801] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:47:24 2022] PUT /identity/v3/projects/eceb00f160194615adc439f1e2bb0e22/users/dfbcad775058475e8a295c3244f2d01b/roles/60af5088793e4379b88821a86552a344 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:47:24.855187 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c43a871-340c-4dfe-b032-a0e971ae2026 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:47:24.865980 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c43a871-340c-4dfe-b032-a0e971ae2026 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:47:24.919752 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8739e01-e4a5-4e6c-9bb7-165f4a2ca253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:47:24.920414 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8739e01-e4a5-4e6c-9bb7-165f4a2ca253 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:47:24.920693 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8739e01-e4a5-4e6c-9bb7-165f4a2ca253 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:47:24.920995 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8739e01-e4a5-4e6c-9bb7-165f4a2ca253 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:47:24.921704 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8739e01-e4a5-4e6c-9bb7-165f4a2ca253 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:47:24.923295 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8739e01-e4a5-4e6c-9bb7-165f4a2ca253 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:47:24.933399 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1402/2802] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:47:24 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:47:24.937543 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2a267f-1ac9-472e-ac5b-ec7a416dc063 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:47:24.947557 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2a267f-1ac9-472e-ac5b-ec7a416dc063 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:47:24.952194 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d66b80-9077-462c-ae6c-87cab7c1e151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:47:24.953069 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d66b80-9077-462c-ae6c-87cab7c1e151 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:47:24.953348 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d66b80-9077-462c-ae6c-87cab7c1e151 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:47:24.953618 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d66b80-9077-462c-ae6c-87cab7c1e151 admin admin] PATH_INFO: `/v3/projects/eceb00f160194615adc439f1e2bb0e22/users/dfbcad775058475e8a295c3244f2d01b/roles/60af5088793e4379b88821a86552a344` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:47:24.955599 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d66b80-9077-462c-ae6c-87cab7c1e151 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eceb00f160194615adc439f1e2bb0e22, user_id=dfbcad775058475e8a295c3244f2d01b, role_id=60af5088793e4379b88821a86552a344)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:47:24.958227 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d66b80-9077-462c-ae6c-87cab7c1e151 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:47:24.968260 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1401/2803] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:47:24 2022] PUT /identity/v3/projects/eceb00f160194615adc439f1e2bb0e22/users/dfbcad775058475e8a295c3244f2d01b/roles/60af5088793e4379b88821a86552a344 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:47:24.983393 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12943835-35d1-47f7-831b-a67b234e0f80 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:47:24.983711 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12943835-35d1-47f7-831b-a67b234e0f80 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:47:24.983988 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12943835-35d1-47f7-831b-a67b234e0f80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:47:25.009749 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-12943835-35d1-47f7-831b-a67b234e0f80 None None] MFA Rules not processed for user `dfbcad775058475e8a295c3244f2d01b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:47:25.010877 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-12943835-35d1-47f7-831b-a67b234e0f80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:47:25.075753 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1403/2804] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:47:24 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:47:25.086843 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d66b80-9077-462c-ae6c-87cab7c1e151 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:47:25.088574 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-74d66b80-9077-462c-ae6c-87cab7c1e151 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:47:25.143234 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d66b80-9077-462c-ae6c-87cab7c1e151 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:47:25.146788 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c990f64-ba3a-4406-b092-24bde92db3d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd05409b38ae346458abfe41ebda1c59e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:47:25.147412 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c990f64-ba3a-4406-b092-24bde92db3d4 service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:47:25.147722 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c990f64-ba3a-4406-b092-24bde92db3d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:47:25.147996 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c990f64-ba3a-4406-b092-24bde92db3d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:47:25.156669 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c990f64-ba3a-4406-b092-24bde92db3d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:47:25.158084 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c990f64-ba3a-4406-b092-24bde92db3d4 service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:47:25.167944 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1402/2805] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:47:25 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:47:25.194870 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12943835-35d1-47f7-831b-a67b234e0f80 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:47:25.254058 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12943835-35d1-47f7-831b-a67b234e0f80 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:47:25.258540 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45b8d19-466d-4f13-b787-82a4df1c27ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:47:25.259421 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45b8d19-466d-4f13-b787-82a4df1c27ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:47:25.259845 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45b8d19-466d-4f13-b787-82a4df1c27ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:47:25.260063 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45b8d19-466d-4f13-b787-82a4df1c27ec admin admin] PATH_INFO: `/v3/users/c56ed194fc3f4b6886fb6d36dcf09e43` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:47:25.261238 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45b8d19-466d-4f13-b787-82a4df1c27ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=c56ed194fc3f4b6886fb6d36dcf09e43)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:47:25.263151 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45b8d19-466d-4f13-b787-82a4df1c27ec admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:47:25.463782 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d45b8d19-466d-4f13-b787-82a4df1c27ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c56ed194fc3f4b6886fb6d36dcf09e43'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:47:25.464546 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d45b8d19-466d-4f13-b787-82a4df1c27ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c56ed194fc3f4b6886fb6d36dcf09e43'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:47:25.476373 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d45b8d19-466d-4f13-b787-82a4df1c27ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c56ed194fc3f4b6886fb6d36dcf09e43'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:47:25.569266 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d45b8d19-466d-4f13-b787-82a4df1c27ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c56ed194fc3f4b6886fb6d36dcf09e43'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:47:25.576375 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d45b8d19-466d-4f13-b787-82a4df1c27ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c56ed194fc3f4b6886fb6d36dcf09e43'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:47:25.582894 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1404/2806] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 17 19:47:25 2022] DELETE /identity/v3/users/c56ed194fc3f4b6886fb6d36dcf09e43 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:47:25.587449 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b7d0da-2fb0-456c-b5a3-36accc96dd4a None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:47:25.587966 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b7d0da-2fb0-456c-b5a3-36accc96dd4a None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:47:25.588095 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b7d0da-2fb0-456c-b5a3-36accc96dd4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:47:25.605153 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-49b7d0da-2fb0-456c-b5a3-36accc96dd4a None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:47:25.606320 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-49b7d0da-2fb0-456c-b5a3-36accc96dd4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:47:25.663113 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1403/2807] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:47:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 17 19:47:25.672803 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45b8d19-466d-4f13-b787-82a4df1c27ec admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:47:25.674187 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d45b8d19-466d-4f13-b787-82a4df1c27ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:47:25.729135 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45b8d19-466d-4f13-b787-82a4df1c27ec admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:47:25.732908 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a77639d-96ec-451b-8f2a-154c4ff303ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:47:25.733541 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a77639d-96ec-451b-8f2a-154c4ff303ea service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:47:25.733817 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a77639d-96ec-451b-8f2a-154c4ff303ea service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:47:25.734091 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a77639d-96ec-451b-8f2a-154c4ff303ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:47:25.742895 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a77639d-96ec-451b-8f2a-154c4ff303ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:47:25.744344 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a77639d-96ec-451b-8f2a-154c4ff303ea service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:47:25.754032 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1405/2808] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:47:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:47:26.178022 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b7d0da-2fb0-456c-b5a3-36accc96dd4a None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:47:26.179432 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-49b7d0da-2fb0-456c-b5a3-36accc96dd4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:47:26.191667 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b7d0da-2fb0-456c-b5a3-36accc96dd4a None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:47:26.196289 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfa2520-fd3a-4679-bcae-3ecd2f2e9f53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:47:26.197142 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfa2520-fd3a-4679-bcae-3ecd2f2e9f53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:47:26.197422 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfa2520-fd3a-4679-bcae-3ecd2f2e9f53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:47:26.197692 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfa2520-fd3a-4679-bcae-3ecd2f2e9f53 admin admin] PATH_INFO: `/v3/projects/ecd4b95cf99340639585f8ea27b69b00` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:47:26.198879 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfa2520-fd3a-4679-bcae-3ecd2f2e9f53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ecd4b95cf99340639585f8ea27b69b00)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:47:26.200815 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfa2520-fd3a-4679-bcae-3ecd2f2e9f53 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:47:26.306989 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-cdfa2520-fd3a-4679-bcae-3ecd2f2e9f53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ecd4b95cf99340639585f8ea27b69b00', 'request_id': 'req-cdfa2520-fd3a-4679-bcae-3ecd2f2e9f53'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:47:26.312960 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-cdfa2520-fd3a-4679-bcae-3ecd2f2e9f53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ecd4b95cf99340639585f8ea27b69b00', 'request_id': 'req-cdfa2520-fd3a-4679-bcae-3ecd2f2e9f53'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:47:26.318873 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-cdfa2520-fd3a-4679-bcae-3ecd2f2e9f53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ecd4b95cf99340639585f8ea27b69b00', 'request_id': 'req-cdfa2520-fd3a-4679-bcae-3ecd2f2e9f53'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:47:26.386241 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-cdfa2520-fd3a-4679-bcae-3ecd2f2e9f53 admin admin] The token cache is being invalidate because project ecd4b95cf99340639585f8ea27b69b00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 17 19:47:26.386527 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-cdfa2520-fd3a-4679-bcae-3ecd2f2e9f53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:47:26.387405 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-cdfa2520-fd3a-4679-bcae-3ecd2f2e9f53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:47:26.389003 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1404/2809] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 17 19:47:26 2022] DELETE /identity/v3/projects/ecd4b95cf99340639585f8ea27b69b00 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:47:26.393035 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a77639d-96ec-451b-8f2a-154c4ff303ea service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:47:26.402623 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a77639d-96ec-451b-8f2a-154c4ff303ea service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:47:26.457572 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c16114-2009-4339-838b-e4c4c10bea32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:47:26.458509 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c16114-2009-4339-838b-e4c4c10bea32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:47:26.458872 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c16114-2009-4339-838b-e4c4c10bea32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:47:26.459140 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c16114-2009-4339-838b-e4c4c10bea32 admin admin] PATH_INFO: `/v3/users/dfbcad775058475e8a295c3244f2d01b` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:47:26.460344 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c16114-2009-4339-838b-e4c4c10bea32 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dfbcad775058475e8a295c3244f2d01b)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:47:26.462358 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c16114-2009-4339-838b-e4c4c10bea32 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:47:26.622246 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-92c16114-2009-4339-838b-e4c4c10bea32 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dfbcad775058475e8a295c3244f2d01b'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:47:26.622898 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-92c16114-2009-4339-838b-e4c4c10bea32 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dfbcad775058475e8a295c3244f2d01b'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:47:26.634142 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-92c16114-2009-4339-838b-e4c4c10bea32 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dfbcad775058475e8a295c3244f2d01b'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:47:26.696251 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-92c16114-2009-4339-838b-e4c4c10bea32 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dfbcad775058475e8a295c3244f2d01b'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:47:26.703030 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-92c16114-2009-4339-838b-e4c4c10bea32 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dfbcad775058475e8a295c3244f2d01b'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:47:26.709404 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1406/2810] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 17 19:47:26 2022] DELETE /identity/v3/users/dfbcad775058475e8a295c3244f2d01b => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 17 19:47:27.132036 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfa2520-fd3a-4679-bcae-3ecd2f2e9f53 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:47:27.133479 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-cdfa2520-fd3a-4679-bcae-3ecd2f2e9f53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:47:27.148470 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfa2520-fd3a-4679-bcae-3ecd2f2e9f53 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:47:27.204518 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be31da01-bb4f-42c2-8485-0cc8ecfda452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:47:27.205996 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be31da01-bb4f-42c2-8485-0cc8ecfda452 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:47:27.206275 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be31da01-bb4f-42c2-8485-0cc8ecfda452 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:47:27.206546 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be31da01-bb4f-42c2-8485-0cc8ecfda452 admin admin] PATH_INFO: `/v3/projects/eceb00f160194615adc439f1e2bb0e22` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:47:27.207738 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be31da01-bb4f-42c2-8485-0cc8ecfda452 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eceb00f160194615adc439f1e2bb0e22)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:47:27.209684 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be31da01-bb4f-42c2-8485-0cc8ecfda452 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:47:27.341191 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-be31da01-bb4f-42c2-8485-0cc8ecfda452 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eceb00f160194615adc439f1e2bb0e22', 'request_id': 'req-be31da01-bb4f-42c2-8485-0cc8ecfda452'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:47:27.347387 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-be31da01-bb4f-42c2-8485-0cc8ecfda452 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eceb00f160194615adc439f1e2bb0e22', 'request_id': 'req-be31da01-bb4f-42c2-8485-0cc8ecfda452'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:47:27.352632 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-be31da01-bb4f-42c2-8485-0cc8ecfda452 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eceb00f160194615adc439f1e2bb0e22', 'request_id': 'req-be31da01-bb4f-42c2-8485-0cc8ecfda452'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:47:27.402978 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-be31da01-bb4f-42c2-8485-0cc8ecfda452 admin admin] The token cache is being invalidate because project eceb00f160194615adc439f1e2bb0e22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 17 19:47:27.403261 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-be31da01-bb4f-42c2-8485-0cc8ecfda452 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:47:27.403893 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-be31da01-bb4f-42c2-8485-0cc8ecfda452 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:47:27.405443 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1405/2811] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 17 19:47:27 2022] DELETE /identity/v3/projects/eceb00f160194615adc439f1e2bb0e22 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:47:27.410505 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0cef11-5793-47ed-bae8-8eae121ce41f None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:47:27.412022 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0cef11-5793-47ed-bae8-8eae121ce41f None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:47:27.412022 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0cef11-5793-47ed-bae8-8eae121ce41f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:47:27.428612 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-ab0cef11-5793-47ed-bae8-8eae121ce41f None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:47:27.429785 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-ab0cef11-5793-47ed-bae8-8eae121ce41f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:47:27.487309 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1407/2812] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:47:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:47:27.495112 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7c172c-eea6-4865-9d38-d6cbc780774a None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:47:27.495399 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7c172c-eea6-4865-9d38-d6cbc780774a None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:47:27.495709 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7c172c-eea6-4865-9d38-d6cbc780774a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:47:27.514027 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-1e7c172c-eea6-4865-9d38-d6cbc780774a None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:47:27.515225 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-1e7c172c-eea6-4865-9d38-d6cbc780774a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:47:27.523030 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1406/2813] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:47:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:47:27.527034 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0cef11-5793-47ed-bae8-8eae121ce41f None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:47:27.536458 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0cef11-5793-47ed-bae8-8eae121ce41f None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:47:27.540882 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ccc3db-df44-4440-b518-218a7e573d54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:47:27.541523 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ccc3db-df44-4440-b518-218a7e573d54 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:47:27.541802 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ccc3db-df44-4440-b518-218a7e573d54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:47:27.542071 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ccc3db-df44-4440-b518-218a7e573d54 admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:47:27.542782 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ccc3db-df44-4440-b518-218a7e573d54 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:47:27.543081 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ccc3db-df44-4440-b518-218a7e573d54 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:47:27.544693 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ccc3db-df44-4440-b518-218a7e573d54 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:47:27.556317 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1408/2814] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 17 19:47:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 17 19:47:27.560404 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7c172c-eea6-4865-9d38-d6cbc780774a None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:47:27.571032 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7c172c-eea6-4865-9d38-d6cbc780774a None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:47:27.575759 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb63dd56-9810-4332-9659-7d9c42753ce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:47:27.576425 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb63dd56-9810-4332-9659-7d9c42753ce9 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:47:27.576705 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb63dd56-9810-4332-9659-7d9c42753ce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:47:27.576981 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb63dd56-9810-4332-9659-7d9c42753ce9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:47:27.577662 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb63dd56-9810-4332-9659-7d9c42753ce9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:47:27.579586 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb63dd56-9810-4332-9659-7d9c42753ce9 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:47:27.708669 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1407/2815] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 17 19:47:27 2022] POST /identity/v3/projects => generated 393 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:47:27.712742 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ccc3db-df44-4440-b518-218a7e573d54 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:47:27.722661 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ccc3db-df44-4440-b518-218a7e573d54 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:47:27.776731 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9002c0d6-491e-41b8-84b3-d8fa0d3980c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:47:27.777379 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9002c0d6-491e-41b8-84b3-d8fa0d3980c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:47:27.777655 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9002c0d6-491e-41b8-84b3-d8fa0d3980c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:47:27.777919 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9002c0d6-491e-41b8-84b3-d8fa0d3980c4 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:47:27.778594 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9002c0d6-491e-41b8-84b3-d8fa0d3980c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:47:27.780530 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9002c0d6-491e-41b8-84b3-d8fa0d3980c4 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:47:27.855590 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1409/2816] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 17 19:47:27 2022] POST /identity/v3/users => generated 297 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:47:27.859661 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb63dd56-9810-4332-9659-7d9c42753ce9 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:47:27.870416 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb63dd56-9810-4332-9659-7d9c42753ce9 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:47:27.875041 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6050fde4-bf21-4af4-b16e-780225f49ca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:47:27.875726 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6050fde4-bf21-4af4-b16e-780225f49ca6 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:47:27.876014 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6050fde4-bf21-4af4-b16e-780225f49ca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:47:27.876289 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6050fde4-bf21-4af4-b16e-780225f49ca6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:47:27.877039 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6050fde4-bf21-4af4-b16e-780225f49ca6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:47:27.878654 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6050fde4-bf21-4af4-b16e-780225f49ca6 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:47:27.888784 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1408/2817] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:47:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:47:27.892711 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9002c0d6-491e-41b8-84b3-d8fa0d3980c4 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:47:27.902737 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9002c0d6-491e-41b8-84b3-d8fa0d3980c4 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:47:27.907473 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f5c7ea-49ef-4d28-a41c-0e7969530b38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:47:27.908183 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f5c7ea-49ef-4d28-a41c-0e7969530b38 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:47:27.908951 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f5c7ea-49ef-4d28-a41c-0e7969530b38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:47:27.909222 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f5c7ea-49ef-4d28-a41c-0e7969530b38 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:47:27.909960 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f5c7ea-49ef-4d28-a41c-0e7969530b38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:47:27.911607 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f5c7ea-49ef-4d28-a41c-0e7969530b38 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:47:27.921239 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1410/2818] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:47:27 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:47:27.925956 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6050fde4-bf21-4af4-b16e-780225f49ca6 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:47:27.935675 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6050fde4-bf21-4af4-b16e-780225f49ca6 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:47:27.940268 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9edc91-aadf-4f4b-b14d-537bae63c2f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:47:27.941127 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9edc91-aadf-4f4b-b14d-537bae63c2f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:47:27.941409 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9edc91-aadf-4f4b-b14d-537bae63c2f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:47:27.941684 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9edc91-aadf-4f4b-b14d-537bae63c2f1 admin admin] PATH_INFO: `/v3/projects/4c8e8c2f127448289ad75fc983758e4b/users/247b67a7fb07426382fa1e68ef0e8f9a/roles/114e6a16e63447d1924707ace5aace25` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:47:27.956047 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9edc91-aadf-4f4b-b14d-537bae63c2f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c8e8c2f127448289ad75fc983758e4b, user_id=247b67a7fb07426382fa1e68ef0e8f9a, role_id=114e6a16e63447d1924707ace5aace25)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:47:27.958741 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9edc91-aadf-4f4b-b14d-537bae63c2f1 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:47:28.021219 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1409/2819] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:47:27 2022] PUT /identity/v3/projects/4c8e8c2f127448289ad75fc983758e4b/users/247b67a7fb07426382fa1e68ef0e8f9a/roles/114e6a16e63447d1924707ace5aace25 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:47:28.032517 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4238a89-4a11-4aaa-9b34-b1ed0fb1382d None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:47:28.032802 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4238a89-4a11-4aaa-9b34-b1ed0fb1382d None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:47:28.033077 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4238a89-4a11-4aaa-9b34-b1ed0fb1382d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:47:28.059956 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-b4238a89-4a11-4aaa-9b34-b1ed0fb1382d None None] MFA Rules not processed for user `247b67a7fb07426382fa1e68ef0e8f9a`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:47:28.061138 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b4238a89-4a11-4aaa-9b34-b1ed0fb1382d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:47:28.122949 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1411/2820] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:47:28 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:47:28.127403 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9edc91-aadf-4f4b-b14d-537bae63c2f1 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:47:28.185828 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9edc91-aadf-4f4b-b14d-537bae63c2f1 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:47:28.190446 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613fd188-9c3d-4d91-9d02-f3aeda133748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:47:28.191079 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613fd188-9c3d-4d91-9d02-f3aeda133748 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:47:28.191353 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613fd188-9c3d-4d91-9d02-f3aeda133748 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:47:28.191650 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613fd188-9c3d-4d91-9d02-f3aeda133748 admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:47:28.192329 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613fd188-9c3d-4d91-9d02-f3aeda133748 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:47:28.194184 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613fd188-9c3d-4d91-9d02-f3aeda133748 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:47:28.262781 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1410/2821] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 17 19:47:28 2022] POST /identity/v3/projects => generated 391 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 17 19:47:28.266746 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4238a89-4a11-4aaa-9b34-b1ed0fb1382d None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:47:28.277620 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4238a89-4a11-4aaa-9b34-b1ed0fb1382d None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:47:28.329970 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4df2e7-c850-48a5-83be-663e27642abe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:47:28.331107 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4df2e7-c850-48a5-83be-663e27642abe admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:47:28.331386 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4df2e7-c850-48a5-83be-663e27642abe admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:47:28.331737 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4df2e7-c850-48a5-83be-663e27642abe admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:47:28.332413 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4df2e7-c850-48a5-83be-663e27642abe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:47:28.334315 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4df2e7-c850-48a5-83be-663e27642abe admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:47:28.433000 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1412/2822] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 17 19:47:28 2022] POST /identity/v3/users => generated 302 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:47:28.436851 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613fd188-9c3d-4d91-9d02-f3aeda133748 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:47:28.446856 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613fd188-9c3d-4d91-9d02-f3aeda133748 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:47:28.451561 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e316d799-3196-4dba-97b0-3520ec0b359b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:47:28.452203 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e316d799-3196-4dba-97b0-3520ec0b359b admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:47:28.452488 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e316d799-3196-4dba-97b0-3520ec0b359b admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:47:28.452765 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e316d799-3196-4dba-97b0-3520ec0b359b admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:47:28.453482 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e316d799-3196-4dba-97b0-3520ec0b359b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:47:28.455576 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e316d799-3196-4dba-97b0-3520ec0b359b admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:47:28.465798 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1411/2823] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:47:28 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:47:28.469606 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4df2e7-c850-48a5-83be-663e27642abe admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:47:28.479867 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4df2e7-c850-48a5-83be-663e27642abe admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:47:28.484612 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405d6461-cf6b-49ad-923b-e80e01c3efe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:47:28.485463 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405d6461-cf6b-49ad-923b-e80e01c3efe1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:47:28.485790 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405d6461-cf6b-49ad-923b-e80e01c3efe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:47:28.486009 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405d6461-cf6b-49ad-923b-e80e01c3efe1 admin admin] PATH_INFO: `/v3/projects/60274b6901ff423baf3d8ee1dbdad255/users/6851a2ff34fa4340a41d3948b3c5275e/roles/60af5088793e4379b88821a86552a344` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:47:28.500002 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405d6461-cf6b-49ad-923b-e80e01c3efe1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60274b6901ff423baf3d8ee1dbdad255, user_id=6851a2ff34fa4340a41d3948b3c5275e, role_id=60af5088793e4379b88821a86552a344)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:47:28.503221 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405d6461-cf6b-49ad-923b-e80e01c3efe1 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:47:28.586639 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1413/2824] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:47:28 2022] PUT /identity/v3/projects/60274b6901ff423baf3d8ee1dbdad255/users/6851a2ff34fa4340a41d3948b3c5275e/roles/60af5088793e4379b88821a86552a344 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:47:28.590514 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e316d799-3196-4dba-97b0-3520ec0b359b admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:47:28.600305 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e316d799-3196-4dba-97b0-3520ec0b359b admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:47:28.654773 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e00397-bf82-4b35-b31e-5c9dbcfe1851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:47:28.655419 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e00397-bf82-4b35-b31e-5c9dbcfe1851 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:47:28.657939 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e00397-bf82-4b35-b31e-5c9dbcfe1851 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:47:28.657939 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e00397-bf82-4b35-b31e-5c9dbcfe1851 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:47:28.657939 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e00397-bf82-4b35-b31e-5c9dbcfe1851 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:47:28.658315 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e00397-bf82-4b35-b31e-5c9dbcfe1851 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:47:28.667642 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1412/2825] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:47:28 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:47:28.671714 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405d6461-cf6b-49ad-923b-e80e01c3efe1 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:47:28.681900 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405d6461-cf6b-49ad-923b-e80e01c3efe1 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:47:28.686496 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b18a32-2354-4674-97d7-5de8f9095257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:47:28.687360 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b18a32-2354-4674-97d7-5de8f9095257 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:47:28.687678 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b18a32-2354-4674-97d7-5de8f9095257 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:47:28.687953 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b18a32-2354-4674-97d7-5de8f9095257 admin admin] PATH_INFO: `/v3/projects/60274b6901ff423baf3d8ee1dbdad255/users/6851a2ff34fa4340a41d3948b3c5275e/roles/60af5088793e4379b88821a86552a344` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:47:28.689946 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b18a32-2354-4674-97d7-5de8f9095257 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60274b6901ff423baf3d8ee1dbdad255, user_id=6851a2ff34fa4340a41d3948b3c5275e, role_id=60af5088793e4379b88821a86552a344)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:47:28.693123 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b18a32-2354-4674-97d7-5de8f9095257 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:47:28.703045 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1414/2826] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:47:28 2022] PUT /identity/v3/projects/60274b6901ff423baf3d8ee1dbdad255/users/6851a2ff34fa4340a41d3948b3c5275e/roles/60af5088793e4379b88821a86552a344 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:47:28.713721 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869fa868-3445-459a-adf5-7309676abb83 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:47:28.714004 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869fa868-3445-459a-adf5-7309676abb83 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:47:28.714276 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869fa868-3445-459a-adf5-7309676abb83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:47:28.740265 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-869fa868-3445-459a-adf5-7309676abb83 None None] MFA Rules not processed for user `6851a2ff34fa4340a41d3948b3c5275e`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:47:28.741403 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-869fa868-3445-459a-adf5-7309676abb83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:47:28.806684 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1413/2827] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:47:28 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:47:28.817832 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b18a32-2354-4674-97d7-5de8f9095257 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:47:28.819625 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-40b18a32-2354-4674-97d7-5de8f9095257 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:47:28.874264 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b18a32-2354-4674-97d7-5de8f9095257 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:47:28.877976 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5fa845-b28c-4a64-9912-1371c2fa978e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd05409b38ae346458abfe41ebda1c59e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:47:28.879099 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5fa845-b28c-4a64-9912-1371c2fa978e service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:47:28.879383 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5fa845-b28c-4a64-9912-1371c2fa978e service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:47:28.879683 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5fa845-b28c-4a64-9912-1371c2fa978e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:47:28.888391 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5fa845-b28c-4a64-9912-1371c2fa978e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:47:28.889815 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5fa845-b28c-4a64-9912-1371c2fa978e service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:47:28.899593 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1415/2828] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:47:28 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:47:28.943305 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869fa868-3445-459a-adf5-7309676abb83 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:47:28.944743 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-869fa868-3445-459a-adf5-7309676abb83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:47:28.956450 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869fa868-3445-459a-adf5-7309676abb83 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:47:28.960055 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642ccade-222e-4165-b164-d019140566cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd05409b38ae346458abfe41ebda1c59e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:47:28.960685 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642ccade-222e-4165-b164-d019140566cf service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:47:28.960968 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642ccade-222e-4165-b164-d019140566cf service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:47:28.961241 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642ccade-222e-4165-b164-d019140566cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:47:29.015515 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642ccade-222e-4165-b164-d019140566cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:47:29.016874 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642ccade-222e-4165-b164-d019140566cf service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:47:29.025527 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1414/2829] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:47:28 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:47:33.597392 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5fa845-b28c-4a64-9912-1371c2fa978e service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:47:33.658502 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5fa845-b28c-4a64-9912-1371c2fa978e service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:47:33.663039 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab916dc-37a1-45f7-91f7-7e2a825eb8c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:47:33.663953 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab916dc-37a1-45f7-91f7-7e2a825eb8c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:47:33.664229 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab916dc-37a1-45f7-91f7-7e2a825eb8c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:47:33.664499 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab916dc-37a1-45f7-91f7-7e2a825eb8c4 admin admin] PATH_INFO: `/v3/users/247b67a7fb07426382fa1e68ef0e8f9a` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:47:33.665687 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab916dc-37a1-45f7-91f7-7e2a825eb8c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=247b67a7fb07426382fa1e68ef0e8f9a)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:47:33.667656 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab916dc-37a1-45f7-91f7-7e2a825eb8c4 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:47:33.903566 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-aab916dc-37a1-45f7-91f7-7e2a825eb8c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '247b67a7fb07426382fa1e68ef0e8f9a'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:47:33.904243 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-aab916dc-37a1-45f7-91f7-7e2a825eb8c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '247b67a7fb07426382fa1e68ef0e8f9a'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:47:33.916230 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-aab916dc-37a1-45f7-91f7-7e2a825eb8c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '247b67a7fb07426382fa1e68ef0e8f9a'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:47:33.981504 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-aab916dc-37a1-45f7-91f7-7e2a825eb8c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '247b67a7fb07426382fa1e68ef0e8f9a'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:47:33.988815 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-aab916dc-37a1-45f7-91f7-7e2a825eb8c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '247b67a7fb07426382fa1e68ef0e8f9a'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:47:33.995207 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1416/2830] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 17 19:47:33 2022] DELETE /identity/v3/users/247b67a7fb07426382fa1e68ef0e8f9a => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:47:33.999794 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d7b36b-fd03-4302-9c95-33e47a137756 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:47:34.000270 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d7b36b-fd03-4302-9c95-33e47a137756 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:47:34.000372 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d7b36b-fd03-4302-9c95-33e47a137756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:47:34.016873 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-a7d7b36b-fd03-4302-9c95-33e47a137756 None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:47:34.018053 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-a7d7b36b-fd03-4302-9c95-33e47a137756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:47:34.074068 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1415/2831] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:47:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:47:34.083750 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab916dc-37a1-45f7-91f7-7e2a825eb8c4 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:47:34.085128 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-aab916dc-37a1-45f7-91f7-7e2a825eb8c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:47:34.139759 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab916dc-37a1-45f7-91f7-7e2a825eb8c4 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:47:34.143305 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8df7222-f218-4b51-9067-a137890fe3fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:47:34.144010 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8df7222-f218-4b51-9067-a137890fe3fb service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:47:34.144287 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8df7222-f218-4b51-9067-a137890fe3fb service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:47:34.144560 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8df7222-f218-4b51-9067-a137890fe3fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:47:34.153771 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8df7222-f218-4b51-9067-a137890fe3fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:47:34.155165 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8df7222-f218-4b51-9067-a137890fe3fb service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:47:34.164849 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1417/2832] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:47:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:47:34.846215 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d7b36b-fd03-4302-9c95-33e47a137756 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:47:34.848004 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-a7d7b36b-fd03-4302-9c95-33e47a137756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:47:34.862566 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d7b36b-fd03-4302-9c95-33e47a137756 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:47:34.869133 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c15780-f4b5-494e-ad99-e91c647e96f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:47:34.869898 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c15780-f4b5-494e-ad99-e91c647e96f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:47:34.870159 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c15780-f4b5-494e-ad99-e91c647e96f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:47:34.870433 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c15780-f4b5-494e-ad99-e91c647e96f4 admin admin] PATH_INFO: `/v3/projects/4c8e8c2f127448289ad75fc983758e4b` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:47:34.871632 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c15780-f4b5-494e-ad99-e91c647e96f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c8e8c2f127448289ad75fc983758e4b)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:47:34.873540 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c15780-f4b5-494e-ad99-e91c647e96f4 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:47:34.987442 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-e0c15780-f4b5-494e-ad99-e91c647e96f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c8e8c2f127448289ad75fc983758e4b', 'request_id': 'req-e0c15780-f4b5-494e-ad99-e91c647e96f4'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:47:34.994089 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-e0c15780-f4b5-494e-ad99-e91c647e96f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c8e8c2f127448289ad75fc983758e4b', 'request_id': 'req-e0c15780-f4b5-494e-ad99-e91c647e96f4'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:47:34.999564 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-e0c15780-f4b5-494e-ad99-e91c647e96f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c8e8c2f127448289ad75fc983758e4b', 'request_id': 'req-e0c15780-f4b5-494e-ad99-e91c647e96f4'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:47:35.083164 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-e0c15780-f4b5-494e-ad99-e91c647e96f4 admin admin] The token cache is being invalidate because project 4c8e8c2f127448289ad75fc983758e4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 17 19:47:35.083832 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-e0c15780-f4b5-494e-ad99-e91c647e96f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:47:35.084183 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-e0c15780-f4b5-494e-ad99-e91c647e96f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:47:35.085724 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1416/2833] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 17 19:47:34 2022] DELETE /identity/v3/projects/4c8e8c2f127448289ad75fc983758e4b => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:47:35.089727 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8df7222-f218-4b51-9067-a137890fe3fb service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:47:35.099208 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8df7222-f218-4b51-9067-a137890fe3fb service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:47:35.155360 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f92189-90d0-403e-a3be-0a9dc4b302f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:47:35.156243 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f92189-90d0-403e-a3be-0a9dc4b302f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:47:35.156520 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f92189-90d0-403e-a3be-0a9dc4b302f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:47:35.156786 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f92189-90d0-403e-a3be-0a9dc4b302f2 admin admin] PATH_INFO: `/v3/users/6851a2ff34fa4340a41d3948b3c5275e` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:47:35.157986 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f92189-90d0-403e-a3be-0a9dc4b302f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6851a2ff34fa4340a41d3948b3c5275e)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:47:35.160002 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f92189-90d0-403e-a3be-0a9dc4b302f2 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:47:35.353510 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-43f92189-90d0-403e-a3be-0a9dc4b302f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6851a2ff34fa4340a41d3948b3c5275e'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:47:35.354107 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-43f92189-90d0-403e-a3be-0a9dc4b302f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6851a2ff34fa4340a41d3948b3c5275e'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:47:35.366476 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-43f92189-90d0-403e-a3be-0a9dc4b302f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6851a2ff34fa4340a41d3948b3c5275e'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:47:35.467017 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-43f92189-90d0-403e-a3be-0a9dc4b302f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6851a2ff34fa4340a41d3948b3c5275e'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:47:35.473918 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-43f92189-90d0-403e-a3be-0a9dc4b302f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6851a2ff34fa4340a41d3948b3c5275e'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:47:35.480224 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1418/2834] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 17 19:47:35 2022] DELETE /identity/v3/users/6851a2ff34fa4340a41d3948b3c5275e => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:47:35.904727 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c15780-f4b5-494e-ad99-e91c647e96f4 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:47:35.907032 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-e0c15780-f4b5-494e-ad99-e91c647e96f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:47:35.924836 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c15780-f4b5-494e-ad99-e91c647e96f4 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:47:35.980921 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b865a05-e39c-4e7f-92c8-9b4f2985e7a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:47:35.981769 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b865a05-e39c-4e7f-92c8-9b4f2985e7a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:47:35.982044 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b865a05-e39c-4e7f-92c8-9b4f2985e7a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:47:35.982577 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b865a05-e39c-4e7f-92c8-9b4f2985e7a5 admin admin] PATH_INFO: `/v3/projects/60274b6901ff423baf3d8ee1dbdad255` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:47:35.983770 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b865a05-e39c-4e7f-92c8-9b4f2985e7a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=60274b6901ff423baf3d8ee1dbdad255)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:47:35.985754 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b865a05-e39c-4e7f-92c8-9b4f2985e7a5 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:47:36.157669 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-2b865a05-e39c-4e7f-92c8-9b4f2985e7a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60274b6901ff423baf3d8ee1dbdad255', 'request_id': 'req-2b865a05-e39c-4e7f-92c8-9b4f2985e7a5'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:47:36.163623 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-2b865a05-e39c-4e7f-92c8-9b4f2985e7a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60274b6901ff423baf3d8ee1dbdad255', 'request_id': 'req-2b865a05-e39c-4e7f-92c8-9b4f2985e7a5'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:47:36.168919 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-2b865a05-e39c-4e7f-92c8-9b4f2985e7a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60274b6901ff423baf3d8ee1dbdad255', 'request_id': 'req-2b865a05-e39c-4e7f-92c8-9b4f2985e7a5'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:47:36.283297 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-2b865a05-e39c-4e7f-92c8-9b4f2985e7a5 admin admin] The token cache is being invalidate because project 60274b6901ff423baf3d8ee1dbdad255 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 17 19:47:36.283613 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-2b865a05-e39c-4e7f-92c8-9b4f2985e7a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:47:36.284223 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-2b865a05-e39c-4e7f-92c8-9b4f2985e7a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:47:36.285792 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1417/2835] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 17 19:47:35 2022] DELETE /identity/v3/projects/60274b6901ff423baf3d8ee1dbdad255 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:47:36.290640 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f67fa02-7ed4-447e-a218-ea16bf4ea5ce None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:47:36.291144 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f67fa02-7ed4-447e-a218-ea16bf4ea5ce None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:47:36.291248 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f67fa02-7ed4-447e-a218-ea16bf4ea5ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:47:36.308074 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-5f67fa02-7ed4-447e-a218-ea16bf4ea5ce None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:47:36.309300 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-5f67fa02-7ed4-447e-a218-ea16bf4ea5ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:47:36.366246 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1419/2836] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:47:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:47:36.373687 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a6e31f-6e14-484a-b590-2bf9ba91d2dd None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:47:36.373982 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a6e31f-6e14-484a-b590-2bf9ba91d2dd None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:47:36.374247 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a6e31f-6e14-484a-b590-2bf9ba91d2dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:47:36.392309 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-e0a6e31f-6e14-484a-b590-2bf9ba91d2dd None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:47:36.393493 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-e0a6e31f-6e14-484a-b590-2bf9ba91d2dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:47:36.401383 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1418/2837] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:47:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:47:36.405302 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f67fa02-7ed4-447e-a218-ea16bf4ea5ce None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:47:36.414612 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f67fa02-7ed4-447e-a218-ea16bf4ea5ce None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:47:36.419103 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac36f82-9979-4576-8344-ca85af6ef78d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:47:36.419708 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac36f82-9979-4576-8344-ca85af6ef78d admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:47:36.419985 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac36f82-9979-4576-8344-ca85af6ef78d admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:47:36.420252 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac36f82-9979-4576-8344-ca85af6ef78d admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:47:36.420955 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac36f82-9979-4576-8344-ca85af6ef78d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:47:36.421254 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac36f82-9979-4576-8344-ca85af6ef78d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:47:36.422831 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac36f82-9979-4576-8344-ca85af6ef78d admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:47:36.434386 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1420/2838] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 17 19:47:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:47:36.438282 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a6e31f-6e14-484a-b590-2bf9ba91d2dd None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:47:36.447824 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a6e31f-6e14-484a-b590-2bf9ba91d2dd None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:47:36.452430 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2df1d1-31fd-4fa2-9450-5aef866e408f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:47:36.453064 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2df1d1-31fd-4fa2-9450-5aef866e408f admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:47:36.453348 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2df1d1-31fd-4fa2-9450-5aef866e408f admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:47:36.453624 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2df1d1-31fd-4fa2-9450-5aef866e408f admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:47:36.454299 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2df1d1-31fd-4fa2-9450-5aef866e408f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:47:36.456226 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2df1d1-31fd-4fa2-9450-5aef866e408f admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:47:36.542278 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1419/2839] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 17 19:47:36 2022] POST /identity/v3/projects => generated 409 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:47:36.546339 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac36f82-9979-4576-8344-ca85af6ef78d admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:47:36.556725 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac36f82-9979-4576-8344-ca85af6ef78d admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:47:36.611154 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19154ab-93d5-4634-b172-adb4080f2b79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:47:36.611836 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19154ab-93d5-4634-b172-adb4080f2b79 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:47:36.612113 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19154ab-93d5-4634-b172-adb4080f2b79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:47:36.612380 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19154ab-93d5-4634-b172-adb4080f2b79 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:47:36.613058 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19154ab-93d5-4634-b172-adb4080f2b79 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:47:36.615604 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19154ab-93d5-4634-b172-adb4080f2b79 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:47:36.717191 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1421/2840] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 17 19:47:36 2022] POST /identity/v3/users => generated 305 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 17 19:47:36.721264 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2df1d1-31fd-4fa2-9450-5aef866e408f admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:47:36.731862 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2df1d1-31fd-4fa2-9450-5aef866e408f admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:47:36.736606 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8001d7-3957-4773-8c14-f75f28492184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:47:36.737241 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8001d7-3957-4773-8c14-f75f28492184 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:47:36.737517 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8001d7-3957-4773-8c14-f75f28492184 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:47:36.737786 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8001d7-3957-4773-8c14-f75f28492184 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:47:36.738487 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8001d7-3957-4773-8c14-f75f28492184 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:47:36.740874 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8001d7-3957-4773-8c14-f75f28492184 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:47:36.751228 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1420/2841] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:47:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:47:36.754957 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19154ab-93d5-4634-b172-adb4080f2b79 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:47:36.764650 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19154ab-93d5-4634-b172-adb4080f2b79 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:47:36.769144 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ab3af5-c11a-456d-936a-a43374d38a36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:47:36.769779 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ab3af5-c11a-456d-936a-a43374d38a36 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:47:36.770054 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ab3af5-c11a-456d-936a-a43374d38a36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:47:36.770325 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ab3af5-c11a-456d-936a-a43374d38a36 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:47:36.771049 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ab3af5-c11a-456d-936a-a43374d38a36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:47:36.772662 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ab3af5-c11a-456d-936a-a43374d38a36 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:47:36.782216 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1422/2842] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:47:36 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:47:36.786214 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8001d7-3957-4773-8c14-f75f28492184 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:47:36.796393 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8001d7-3957-4773-8c14-f75f28492184 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:47:36.800962 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a04030-4324-4b86-a8e4-72b30f5c333d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:47:36.801810 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a04030-4324-4b86-a8e4-72b30f5c333d admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:47:36.802086 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a04030-4324-4b86-a8e4-72b30f5c333d admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:47:36.803008 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a04030-4324-4b86-a8e4-72b30f5c333d admin admin] PATH_INFO: `/v3/projects/4596ec6eefe740a98cff6157cb390d0a/users/c4f088f96a084b8aa253c452a5289cb4/roles/114e6a16e63447d1924707ace5aace25` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:47:36.816864 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a04030-4324-4b86-a8e4-72b30f5c333d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4596ec6eefe740a98cff6157cb390d0a, user_id=c4f088f96a084b8aa253c452a5289cb4, role_id=114e6a16e63447d1924707ace5aace25)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:47:36.819549 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a04030-4324-4b86-a8e4-72b30f5c333d admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:47:36.901872 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1421/2843] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:47:36 2022] PUT /identity/v3/projects/4596ec6eefe740a98cff6157cb390d0a/users/c4f088f96a084b8aa253c452a5289cb4/roles/114e6a16e63447d1924707ace5aace25 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:47:36.913585 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57df4cae-524a-494c-b5b0-2370e1efaaf5 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:47:36.913876 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57df4cae-524a-494c-b5b0-2370e1efaaf5 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:47:36.914168 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57df4cae-524a-494c-b5b0-2370e1efaaf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:47:36.940163 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-57df4cae-524a-494c-b5b0-2370e1efaaf5 None None] MFA Rules not processed for user `c4f088f96a084b8aa253c452a5289cb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:47:36.941350 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-57df4cae-524a-494c-b5b0-2370e1efaaf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:47:37.003881 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1423/2844] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:47:36 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:47:37.008316 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a04030-4324-4b86-a8e4-72b30f5c333d admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:47:37.067451 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a04030-4324-4b86-a8e4-72b30f5c333d admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:47:37.072150 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f420f0af-c3da-446d-b693-f631e85e6173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:47:37.072793 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f420f0af-c3da-446d-b693-f631e85e6173 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:47:37.073076 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f420f0af-c3da-446d-b693-f631e85e6173 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:47:37.073352 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f420f0af-c3da-446d-b693-f631e85e6173 admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:47:37.074042 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f420f0af-c3da-446d-b693-f631e85e6173 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:47:37.076006 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f420f0af-c3da-446d-b693-f631e85e6173 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:47:37.159292 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1422/2845] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 17 19:47:37 2022] POST /identity/v3/projects => generated 409 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:47:37.163935 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57df4cae-524a-494c-b5b0-2370e1efaaf5 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:47:37.174740 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57df4cae-524a-494c-b5b0-2370e1efaaf5 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:47:37.228154 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b70c51-6469-4605-a9e8-930360ea0d40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:47:37.228809 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b70c51-6469-4605-a9e8-930360ea0d40 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:47:37.229089 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b70c51-6469-4605-a9e8-930360ea0d40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:47:37.229364 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b70c51-6469-4605-a9e8-930360ea0d40 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:47:37.230044 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b70c51-6469-4605-a9e8-930360ea0d40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:47:37.231983 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b70c51-6469-4605-a9e8-930360ea0d40 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:47:37.329980 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1424/2846] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 17 19:47:37 2022] POST /identity/v3/users => generated 311 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:47:37.334363 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f420f0af-c3da-446d-b693-f631e85e6173 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:47:37.344437 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f420f0af-c3da-446d-b693-f631e85e6173 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:47:37.348955 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7405f7-637a-4cbf-ba5e-c4f2d8f60ed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:47:37.350291 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7405f7-637a-4cbf-ba5e-c4f2d8f60ed4 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:47:37.350570 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7405f7-637a-4cbf-ba5e-c4f2d8f60ed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:47:37.350837 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7405f7-637a-4cbf-ba5e-c4f2d8f60ed4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:47:37.351575 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7405f7-637a-4cbf-ba5e-c4f2d8f60ed4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:47:37.353149 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7405f7-637a-4cbf-ba5e-c4f2d8f60ed4 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:47:37.363529 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1423/2847] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:47:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:47:37.367389 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b70c51-6469-4605-a9e8-930360ea0d40 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:47:37.377044 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b70c51-6469-4605-a9e8-930360ea0d40 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:47:37.381894 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50561ae-9736-4159-9e62-3647bfbe5c58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:47:37.382554 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50561ae-9736-4159-9e62-3647bfbe5c58 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:47:37.382831 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50561ae-9736-4159-9e62-3647bfbe5c58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:47:37.383102 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50561ae-9736-4159-9e62-3647bfbe5c58 admin admin] PATH_INFO: `/v3/projects/21d880faae45457981ec35d4f8ef9a7b/users/72b0abbbe557466280bddea1f6cd169f/roles/60af5088793e4379b88821a86552a344` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:47:37.397740 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50561ae-9736-4159-9e62-3647bfbe5c58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21d880faae45457981ec35d4f8ef9a7b, user_id=72b0abbbe557466280bddea1f6cd169f, role_id=60af5088793e4379b88821a86552a344)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:47:37.400416 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50561ae-9736-4159-9e62-3647bfbe5c58 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:47:37.472352 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1425/2848] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:47:37 2022] PUT /identity/v3/projects/21d880faae45457981ec35d4f8ef9a7b/users/72b0abbbe557466280bddea1f6cd169f/roles/60af5088793e4379b88821a86552a344 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 17 19:47:37.476789 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7405f7-637a-4cbf-ba5e-c4f2d8f60ed4 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:47:37.486687 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7405f7-637a-4cbf-ba5e-c4f2d8f60ed4 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:47:37.540639 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c61894-6655-4447-9ee5-3feaf823f601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:47:37.541293 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c61894-6655-4447-9ee5-3feaf823f601 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:47:37.541570 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c61894-6655-4447-9ee5-3feaf823f601 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:47:37.541840 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c61894-6655-4447-9ee5-3feaf823f601 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:47:37.542547 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c61894-6655-4447-9ee5-3feaf823f601 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:47:37.544167 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c61894-6655-4447-9ee5-3feaf823f601 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:47:37.553408 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1424/2849] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:47:37 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:47:37.557478 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50561ae-9736-4159-9e62-3647bfbe5c58 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:47:37.568553 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50561ae-9736-4159-9e62-3647bfbe5c58 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:47:37.573276 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca539fb-4bc4-4b88-bba9-a1c6690afa43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:47:37.574186 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca539fb-4bc4-4b88-bba9-a1c6690afa43 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:47:37.574466 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca539fb-4bc4-4b88-bba9-a1c6690afa43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:47:37.574736 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca539fb-4bc4-4b88-bba9-a1c6690afa43 admin admin] PATH_INFO: `/v3/projects/21d880faae45457981ec35d4f8ef9a7b/users/72b0abbbe557466280bddea1f6cd169f/roles/60af5088793e4379b88821a86552a344` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:47:37.576854 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca539fb-4bc4-4b88-bba9-a1c6690afa43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21d880faae45457981ec35d4f8ef9a7b, user_id=72b0abbbe557466280bddea1f6cd169f, role_id=60af5088793e4379b88821a86552a344)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:47:37.579599 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca539fb-4bc4-4b88-bba9-a1c6690afa43 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:47:37.590428 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1426/2850] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:47:37 2022] PUT /identity/v3/projects/21d880faae45457981ec35d4f8ef9a7b/users/72b0abbbe557466280bddea1f6cd169f/roles/60af5088793e4379b88821a86552a344 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:47:37.600439 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e20458-4a4e-4d9d-861a-09e3d98af373 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:47:37.600727 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e20458-4a4e-4d9d-861a-09e3d98af373 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:47:37.601004 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e20458-4a4e-4d9d-861a-09e3d98af373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:47:37.626148 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-a1e20458-4a4e-4d9d-861a-09e3d98af373 None None] MFA Rules not processed for user `72b0abbbe557466280bddea1f6cd169f`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:47:37.627339 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-a1e20458-4a4e-4d9d-861a-09e3d98af373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:47:37.692589 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1425/2851] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:47:37 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:47:37.703634 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca539fb-4bc4-4b88-bba9-a1c6690afa43 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:47:37.705371 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-4ca539fb-4bc4-4b88-bba9-a1c6690afa43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:47:37.760428 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca539fb-4bc4-4b88-bba9-a1c6690afa43 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:47:37.764031 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6728207-de5d-4590-964c-d2d79b7fd674 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd05409b38ae346458abfe41ebda1c59e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:47:37.764660 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6728207-de5d-4590-964c-d2d79b7fd674 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:47:37.764939 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6728207-de5d-4590-964c-d2d79b7fd674 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:47:37.765212 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6728207-de5d-4590-964c-d2d79b7fd674 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:47:37.774491 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6728207-de5d-4590-964c-d2d79b7fd674 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:47:37.775948 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6728207-de5d-4590-964c-d2d79b7fd674 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:47:37.785020 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1427/2852] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:47:37 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:47:38.176525 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e20458-4a4e-4d9d-861a-09e3d98af373 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:47:38.186695 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e20458-4a4e-4d9d-861a-09e3d98af373 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:47:38.190300 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c443c8b-72b7-4471-bb88-048a7af3cd9b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd05409b38ae346458abfe41ebda1c59e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:47:38.190927 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c443c8b-72b7-4471-bb88-048a7af3cd9b service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:47:38.191204 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c443c8b-72b7-4471-bb88-048a7af3cd9b service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:47:38.191472 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c443c8b-72b7-4471-bb88-048a7af3cd9b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:47:38.245770 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c443c8b-72b7-4471-bb88-048a7af3cd9b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:47:38.247121 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c443c8b-72b7-4471-bb88-048a7af3cd9b service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:47:38.255663 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1426/2853] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:47:38 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:47:47.994835 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6728207-de5d-4590-964c-d2d79b7fd674 service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:47:48.055624 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6728207-de5d-4590-964c-d2d79b7fd674 service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:47:48.060697 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d28737a-506c-4232-a3db-2fafe4e70f81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:47:48.061550 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d28737a-506c-4232-a3db-2fafe4e70f81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:47:48.061824 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d28737a-506c-4232-a3db-2fafe4e70f81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:47:48.062098 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d28737a-506c-4232-a3db-2fafe4e70f81 admin admin] PATH_INFO: `/v3/users/c4f088f96a084b8aa253c452a5289cb4` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:47:48.063262 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d28737a-506c-4232-a3db-2fafe4e70f81 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4f088f96a084b8aa253c452a5289cb4)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:47:48.065243 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d28737a-506c-4232-a3db-2fafe4e70f81 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:47:48.250484 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1d28737a-506c-4232-a3db-2fafe4e70f81 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4f088f96a084b8aa253c452a5289cb4'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:47:48.251065 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1d28737a-506c-4232-a3db-2fafe4e70f81 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4f088f96a084b8aa253c452a5289cb4'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:47:48.262804 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1d28737a-506c-4232-a3db-2fafe4e70f81 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4f088f96a084b8aa253c452a5289cb4'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:47:48.352306 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1d28737a-506c-4232-a3db-2fafe4e70f81 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4f088f96a084b8aa253c452a5289cb4'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:47:48.359547 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1d28737a-506c-4232-a3db-2fafe4e70f81 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4f088f96a084b8aa253c452a5289cb4'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:47:48.365329 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1428/2854] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 17 19:47:47 2022] DELETE /identity/v3/users/c4f088f96a084b8aa253c452a5289cb4 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:47:48.369889 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55135847-26e4-4304-977b-df2f7f793e1c None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:47:48.370354 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55135847-26e4-4304-977b-df2f7f793e1c None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:47:48.370494 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55135847-26e4-4304-977b-df2f7f793e1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:47:48.387288 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-55135847-26e4-4304-977b-df2f7f793e1c None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:47:48.389197 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-55135847-26e4-4304-977b-df2f7f793e1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:47:48.444859 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1427/2855] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:47:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:47:48.454804 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d28737a-506c-4232-a3db-2fafe4e70f81 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:47:48.456256 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-1d28737a-506c-4232-a3db-2fafe4e70f81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:47:48.512628 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d28737a-506c-4232-a3db-2fafe4e70f81 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:47:48.516275 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef4e3ad-6333-4d95-8acd-f7b38c0c156f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:47:48.517703 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef4e3ad-6333-4d95-8acd-f7b38c0c156f service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:47:48.517703 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef4e3ad-6333-4d95-8acd-f7b38c0c156f service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:47:48.517703 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef4e3ad-6333-4d95-8acd-f7b38c0c156f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:47:48.526926 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef4e3ad-6333-4d95-8acd-f7b38c0c156f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:47:48.528380 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef4e3ad-6333-4d95-8acd-f7b38c0c156f service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:47:48.537777 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1429/2856] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:47:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:47:49.062588 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55135847-26e4-4304-977b-df2f7f793e1c None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:47:49.064109 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-55135847-26e4-4304-977b-df2f7f793e1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:47:49.076245 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55135847-26e4-4304-977b-df2f7f793e1c None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:47:49.081126 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276b02db-20be-4b8b-8b06-154ef5dd8a6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:47:49.082065 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276b02db-20be-4b8b-8b06-154ef5dd8a6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:47:49.082341 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276b02db-20be-4b8b-8b06-154ef5dd8a6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:47:49.082648 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276b02db-20be-4b8b-8b06-154ef5dd8a6e admin admin] PATH_INFO: `/v3/projects/4596ec6eefe740a98cff6157cb390d0a` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:47:49.083859 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276b02db-20be-4b8b-8b06-154ef5dd8a6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=4596ec6eefe740a98cff6157cb390d0a)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:47:49.085774 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276b02db-20be-4b8b-8b06-154ef5dd8a6e admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:47:49.232018 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-276b02db-20be-4b8b-8b06-154ef5dd8a6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4596ec6eefe740a98cff6157cb390d0a', 'request_id': 'req-276b02db-20be-4b8b-8b06-154ef5dd8a6e'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:47:49.238308 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-276b02db-20be-4b8b-8b06-154ef5dd8a6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4596ec6eefe740a98cff6157cb390d0a', 'request_id': 'req-276b02db-20be-4b8b-8b06-154ef5dd8a6e'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:47:49.243671 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-276b02db-20be-4b8b-8b06-154ef5dd8a6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4596ec6eefe740a98cff6157cb390d0a', 'request_id': 'req-276b02db-20be-4b8b-8b06-154ef5dd8a6e'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:47:49.332905 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-276b02db-20be-4b8b-8b06-154ef5dd8a6e admin admin] The token cache is being invalidate because project 4596ec6eefe740a98cff6157cb390d0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 17 19:47:49.333176 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-276b02db-20be-4b8b-8b06-154ef5dd8a6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:47:49.333695 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-276b02db-20be-4b8b-8b06-154ef5dd8a6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:47:49.335273 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1428/2857] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 17 19:47:49 2022] DELETE /identity/v3/projects/4596ec6eefe740a98cff6157cb390d0a => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:47:49.339292 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef4e3ad-6333-4d95-8acd-f7b38c0c156f service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:47:49.348810 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef4e3ad-6333-4d95-8acd-f7b38c0c156f service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:47:49.404576 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790c2672-498c-4edd-8cd5-5c5978f883e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:47:49.405422 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790c2672-498c-4edd-8cd5-5c5978f883e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:47:49.405698 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790c2672-498c-4edd-8cd5-5c5978f883e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:47:49.405966 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790c2672-498c-4edd-8cd5-5c5978f883e6 admin admin] PATH_INFO: `/v3/users/72b0abbbe557466280bddea1f6cd169f` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:47:49.407113 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790c2672-498c-4edd-8cd5-5c5978f883e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=72b0abbbe557466280bddea1f6cd169f)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:47:49.409036 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790c2672-498c-4edd-8cd5-5c5978f883e6 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:47:49.657687 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-790c2672-498c-4edd-8cd5-5c5978f883e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72b0abbbe557466280bddea1f6cd169f'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:47:49.658257 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-790c2672-498c-4edd-8cd5-5c5978f883e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72b0abbbe557466280bddea1f6cd169f'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:47:49.670215 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-790c2672-498c-4edd-8cd5-5c5978f883e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72b0abbbe557466280bddea1f6cd169f'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:47:49.797787 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-790c2672-498c-4edd-8cd5-5c5978f883e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72b0abbbe557466280bddea1f6cd169f'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:47:49.805040 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-790c2672-498c-4edd-8cd5-5c5978f883e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72b0abbbe557466280bddea1f6cd169f'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:47:49.810966 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1430/2858] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 17 19:47:49 2022] DELETE /identity/v3/users/72b0abbbe557466280bddea1f6cd169f => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:47:50.215389 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276b02db-20be-4b8b-8b06-154ef5dd8a6e admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:47:50.218335 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-276b02db-20be-4b8b-8b06-154ef5dd8a6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:47:50.230167 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276b02db-20be-4b8b-8b06-154ef5dd8a6e admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:47:50.285010 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d85f894-3149-4096-b07d-64f3f5589f5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:47:50.285861 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d85f894-3149-4096-b07d-64f3f5589f5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:47:50.286138 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d85f894-3149-4096-b07d-64f3f5589f5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:47:50.286407 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d85f894-3149-4096-b07d-64f3f5589f5c admin admin] PATH_INFO: `/v3/projects/21d880faae45457981ec35d4f8ef9a7b` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:47:50.287609 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d85f894-3149-4096-b07d-64f3f5589f5c admin admin] RBAC: Authorizing `identity:delete_project(project_id=21d880faae45457981ec35d4f8ef9a7b)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:47:50.289499 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d85f894-3149-4096-b07d-64f3f5589f5c admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:47:50.414282 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-4d85f894-3149-4096-b07d-64f3f5589f5c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21d880faae45457981ec35d4f8ef9a7b', 'request_id': 'req-4d85f894-3149-4096-b07d-64f3f5589f5c'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:47:50.420202 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-4d85f894-3149-4096-b07d-64f3f5589f5c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21d880faae45457981ec35d4f8ef9a7b', 'request_id': 'req-4d85f894-3149-4096-b07d-64f3f5589f5c'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:47:50.425442 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-4d85f894-3149-4096-b07d-64f3f5589f5c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21d880faae45457981ec35d4f8ef9a7b', 'request_id': 'req-4d85f894-3149-4096-b07d-64f3f5589f5c'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:47:50.540347 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-4d85f894-3149-4096-b07d-64f3f5589f5c admin admin] The token cache is being invalidate because project 21d880faae45457981ec35d4f8ef9a7b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 17 19:47:50.540567 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-4d85f894-3149-4096-b07d-64f3f5589f5c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:47:50.541181 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-4d85f894-3149-4096-b07d-64f3f5589f5c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:47:50.543215 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1429/2859] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 17 19:47:50 2022] DELETE /identity/v3/projects/21d880faae45457981ec35d4f8ef9a7b => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:47:50.548192 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eddce2-938f-45c0-8336-6a1b448a70cb None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:47:50.548669 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eddce2-938f-45c0-8336-6a1b448a70cb None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:47:50.548783 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eddce2-938f-45c0-8336-6a1b448a70cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:47:50.566607 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-a9eddce2-938f-45c0-8336-6a1b448a70cb None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:47:50.567849 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a9eddce2-938f-45c0-8336-6a1b448a70cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:47:50.625160 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1431/2860] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:47:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:47:50.634103 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d8422b-a3fe-4410-91d7-ce608ac9fa8e None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:47:50.634392 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d8422b-a3fe-4410-91d7-ce608ac9fa8e None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:47:50.634671 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d8422b-a3fe-4410-91d7-ce608ac9fa8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:47:50.652986 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-55d8422b-a3fe-4410-91d7-ce608ac9fa8e None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:47:50.654155 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-55d8422b-a3fe-4410-91d7-ce608ac9fa8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:47:50.662074 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1430/2861] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:47:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:47:50.666236 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9eddce2-938f-45c0-8336-6a1b448a70cb None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:47:50.676338 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9eddce2-938f-45c0-8336-6a1b448a70cb None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:47:50.681137 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfc008b-dcf1-406f-b557-cb77e49a9010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:47:50.681760 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfc008b-dcf1-406f-b557-cb77e49a9010 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:47:50.682034 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfc008b-dcf1-406f-b557-cb77e49a9010 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:47:50.682300 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfc008b-dcf1-406f-b557-cb77e49a9010 admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:47:50.683040 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfc008b-dcf1-406f-b557-cb77e49a9010 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 17 19:47:50.683988 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfc008b-dcf1-406f-b557-cb77e49a9010 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:47:50.685726 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfc008b-dcf1-406f-b557-cb77e49a9010 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:47:50.700710 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1432/2862] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 17 19:47:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 17 19:47:50.704787 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d8422b-a3fe-4410-91d7-ce608ac9fa8e None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:47:50.713774 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d8422b-a3fe-4410-91d7-ce608ac9fa8e None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:47:50.718449 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad1cce5-3471-4130-8dba-0151f5e5856d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:47:50.719087 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad1cce5-3471-4130-8dba-0151f5e5856d admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:47:50.719361 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad1cce5-3471-4130-8dba-0151f5e5856d admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:47:50.719659 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad1cce5-3471-4130-8dba-0151f5e5856d admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:47:50.720448 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad1cce5-3471-4130-8dba-0151f5e5856d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:47:50.722348 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad1cce5-3471-4130-8dba-0151f5e5856d admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:47:50.817666 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1431/2863] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 17 19:47:50 2022] POST /identity/v3/projects => generated 371 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:47:50.821485 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfc008b-dcf1-406f-b557-cb77e49a9010 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:47:50.831225 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfc008b-dcf1-406f-b557-cb77e49a9010 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:47:50.886086 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302558f2-7369-4209-9599-236a20cc0553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:47:50.886742 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302558f2-7369-4209-9599-236a20cc0553 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:47:50.887017 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302558f2-7369-4209-9599-236a20cc0553 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:47:50.887287 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302558f2-7369-4209-9599-236a20cc0553 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:47:50.888069 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302558f2-7369-4209-9599-236a20cc0553 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:47:50.890033 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302558f2-7369-4209-9599-236a20cc0553 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:47:50.966610 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1433/2864] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 17 19:47:50 2022] POST /identity/v3/users => generated 286 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:47:50.968488 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad1cce5-3471-4130-8dba-0151f5e5856d admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:47:50.979005 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad1cce5-3471-4130-8dba-0151f5e5856d admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:47:50.984629 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf2449f-4adb-43fa-8ebf-7b46f8ae2902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:47:50.985275 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf2449f-4adb-43fa-8ebf-7b46f8ae2902 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:47:50.985549 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf2449f-4adb-43fa-8ebf-7b46f8ae2902 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:47:50.985928 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf2449f-4adb-43fa-8ebf-7b46f8ae2902 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:47:50.986640 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf2449f-4adb-43fa-8ebf-7b46f8ae2902 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:47:50.988255 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf2449f-4adb-43fa-8ebf-7b46f8ae2902 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:47:50.998646 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1432/2865] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:47:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:47:51.006163 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302558f2-7369-4209-9599-236a20cc0553 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:47:51.016173 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302558f2-7369-4209-9599-236a20cc0553 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:47:51.021027 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577f65db-caab-4837-b4db-b09239ec5ffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:47:51.021945 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577f65db-caab-4837-b4db-b09239ec5ffc admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:47:51.022352 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577f65db-caab-4837-b4db-b09239ec5ffc admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:47:51.022684 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577f65db-caab-4837-b4db-b09239ec5ffc admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:47:51.023396 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577f65db-caab-4837-b4db-b09239ec5ffc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:47:51.025482 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577f65db-caab-4837-b4db-b09239ec5ffc admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:47:51.036344 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1434/2866] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:47:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 17 19:47:51.040342 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf2449f-4adb-43fa-8ebf-7b46f8ae2902 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:47:51.050886 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf2449f-4adb-43fa-8ebf-7b46f8ae2902 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:47:51.055632 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8534357-637b-4a8a-b7ff-1584c74e2b01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:47:51.056518 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8534357-637b-4a8a-b7ff-1584c74e2b01 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:47:51.056790 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8534357-637b-4a8a-b7ff-1584c74e2b01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:47:51.057056 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8534357-637b-4a8a-b7ff-1584c74e2b01 admin admin] PATH_INFO: `/v3/projects/757859fa777e4b5dad0105d36b6554d1/users/02ec33ef01904b21a2727e861735fc65/roles/114e6a16e63447d1924707ace5aace25` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:47:51.070859 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8534357-637b-4a8a-b7ff-1584c74e2b01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=757859fa777e4b5dad0105d36b6554d1, user_id=02ec33ef01904b21a2727e861735fc65, role_id=114e6a16e63447d1924707ace5aace25)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:47:51.073456 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8534357-637b-4a8a-b7ff-1584c74e2b01 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:47:51.173950 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1433/2867] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:47:51 2022] PUT /identity/v3/projects/757859fa777e4b5dad0105d36b6554d1/users/02ec33ef01904b21a2727e861735fc65/roles/114e6a16e63447d1924707ace5aace25 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 17 19:47:51.179158 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c72964d-9e2f-413d-b925-f8472faab4bb None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:47:51.179451 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c72964d-9e2f-413d-b925-f8472faab4bb None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:47:51.179776 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c72964d-9e2f-413d-b925-f8472faab4bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:47:51.197026 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-8c72964d-9e2f-413d-b925-f8472faab4bb None None] MFA Rules not processed for user `ac2441e330cd41aaa04d125461331f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:47:51.198204 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-8c72964d-9e2f-413d-b925-f8472faab4bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:47:51.254938 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1435/2868] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:47:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:47:51.264682 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8534357-637b-4a8a-b7ff-1584c74e2b01 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:47:51.266350 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-d8534357-637b-4a8a-b7ff-1584c74e2b01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:47:51.321420 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8534357-637b-4a8a-b7ff-1584c74e2b01 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:47:51.325673 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1cf98b-3758-4f88-80d7-0e98db38c288 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:47:51.326306 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1cf98b-3758-4f88-80d7-0e98db38c288 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:47:51.326643 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1cf98b-3758-4f88-80d7-0e98db38c288 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:47:51.326920 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1cf98b-3758-4f88-80d7-0e98db38c288 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:47:51.335841 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1cf98b-3758-4f88-80d7-0e98db38c288 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:47:51.337259 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1cf98b-3758-4f88-80d7-0e98db38c288 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:47:51.347145 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1434/2869] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:47:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:47:56.177854 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5711198-4abd-4741-a5cc-132c740ded2f None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:47:56.178380 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5711198-4abd-4741-a5cc-132c740ded2f None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:47:56.178483 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5711198-4abd-4741-a5cc-132c740ded2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:47:56.209478 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-a5711198-4abd-4741-a5cc-132c740ded2f None None] MFA Rules not processed for user `02ec33ef01904b21a2727e861735fc65`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:47:56.210699 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a5711198-4abd-4741-a5cc-132c740ded2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:47:56.279784 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1436/2870] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:47:56 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:47:56.284974 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1cf98b-3758-4f88-80d7-0e98db38c288 service neutron] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:47:56.295935 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1cf98b-3758-4f88-80d7-0e98db38c288 service neutron] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:47:56.300592 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bee5c1-ec6a-4e1d-a13e-2d8b8acba0d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:47:56.301333 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bee5c1-ec6a-4e1d-a13e-2d8b8acba0d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:47:56.301701 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bee5c1-ec6a-4e1d-a13e-2d8b8acba0d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:47:56.301984 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bee5c1-ec6a-4e1d-a13e-2d8b8acba0d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:47:56.302685 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bee5c1-ec6a-4e1d-a13e-2d8b8acba0d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:47:56.304805 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bee5c1-ec6a-4e1d-a13e-2d8b8acba0d6 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:47:56.394280 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1435/2871] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 17 19:47:56 2022] POST /identity/v3/projects => generated 371 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:47:56.398360 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5711198-4abd-4741-a5cc-132c740ded2f None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:47:56.409657 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5711198-4abd-4741-a5cc-132c740ded2f None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:47:56.468352 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3570f7-3ecf-482b-bc4d-695a4bc78e18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:47:56.469294 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3570f7-3ecf-482b-bc4d-695a4bc78e18 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:47:56.469736 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3570f7-3ecf-482b-bc4d-695a4bc78e18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:47:56.470198 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3570f7-3ecf-482b-bc4d-695a4bc78e18 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:47:56.471056 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3570f7-3ecf-482b-bc4d-695a4bc78e18 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:47:56.473025 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3570f7-3ecf-482b-bc4d-695a4bc78e18 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:47:56.593646 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1437/2872] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 17 19:47:56 2022] POST /identity/v3/users => generated 292 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 17 19:47:56.597573 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bee5c1-ec6a-4e1d-a13e-2d8b8acba0d6 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:47:56.608416 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bee5c1-ec6a-4e1d-a13e-2d8b8acba0d6 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:47:56.613121 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921d2cf9-b12f-43f2-9bad-c35d1a67d8a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:47:56.613776 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921d2cf9-b12f-43f2-9bad-c35d1a67d8a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:47:56.614099 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921d2cf9-b12f-43f2-9bad-c35d1a67d8a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:47:56.614372 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921d2cf9-b12f-43f2-9bad-c35d1a67d8a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:47:56.615090 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921d2cf9-b12f-43f2-9bad-c35d1a67d8a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:47:56.616714 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921d2cf9-b12f-43f2-9bad-c35d1a67d8a0 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:47:56.628229 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1436/2873] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:47:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 17 19:47:56.632321 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3570f7-3ecf-482b-bc4d-695a4bc78e18 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:47:56.645737 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3570f7-3ecf-482b-bc4d-695a4bc78e18 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:47:56.650552 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db23d4be-6daf-4b11-9707-f8ffb894e81a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:47:56.651444 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db23d4be-6daf-4b11-9707-f8ffb894e81a admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:47:56.651832 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db23d4be-6daf-4b11-9707-f8ffb894e81a admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:47:56.652109 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db23d4be-6daf-4b11-9707-f8ffb894e81a admin admin] PATH_INFO: `/v3/projects/c4e2d65fdc32405ba21ff6081ce969d5/users/6cf7c6fa0089421e92ce4e8d030af7ab/roles/60af5088793e4379b88821a86552a344` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:47:56.668728 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db23d4be-6daf-4b11-9707-f8ffb894e81a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4e2d65fdc32405ba21ff6081ce969d5, user_id=6cf7c6fa0089421e92ce4e8d030af7ab, role_id=60af5088793e4379b88821a86552a344)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:47:56.671536 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db23d4be-6daf-4b11-9707-f8ffb894e81a admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:47:56.746108 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1438/2874] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:47:56 2022] PUT /identity/v3/projects/c4e2d65fdc32405ba21ff6081ce969d5/users/6cf7c6fa0089421e92ce4e8d030af7ab/roles/60af5088793e4379b88821a86552a344 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:47:56.750146 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921d2cf9-b12f-43f2-9bad-c35d1a67d8a0 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:47:56.760748 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921d2cf9-b12f-43f2-9bad-c35d1a67d8a0 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:47:56.817622 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497bf99d-ef34-4c54-995c-ee580e804759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:47:56.818287 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497bf99d-ef34-4c54-995c-ee580e804759 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:47:56.818561 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497bf99d-ef34-4c54-995c-ee580e804759 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:47:56.818828 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497bf99d-ef34-4c54-995c-ee580e804759 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:47:56.819572 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497bf99d-ef34-4c54-995c-ee580e804759 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:47:56.821169 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497bf99d-ef34-4c54-995c-ee580e804759 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:47:56.831639 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1437/2875] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 17 19:47:56 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 17 19:47:56.836042 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db23d4be-6daf-4b11-9707-f8ffb894e81a admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:47:56.847230 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db23d4be-6daf-4b11-9707-f8ffb894e81a admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:47:56.852015 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd3038f-5de6-4d94-a53f-8285c200b547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:47:56.852902 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd3038f-5de6-4d94-a53f-8285c200b547 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:47:56.853177 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd3038f-5de6-4d94-a53f-8285c200b547 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:47:56.853445 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd3038f-5de6-4d94-a53f-8285c200b547 admin admin] PATH_INFO: `/v3/projects/c4e2d65fdc32405ba21ff6081ce969d5/users/6cf7c6fa0089421e92ce4e8d030af7ab/roles/60af5088793e4379b88821a86552a344` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:47:56.855547 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd3038f-5de6-4d94-a53f-8285c200b547 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4e2d65fdc32405ba21ff6081ce969d5, user_id=6cf7c6fa0089421e92ce4e8d030af7ab, role_id=60af5088793e4379b88821a86552a344)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:47:56.858577 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd3038f-5de6-4d94-a53f-8285c200b547 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:47:56.870051 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1439/2876] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 17 19:47:56 2022] PUT /identity/v3/projects/c4e2d65fdc32405ba21ff6081ce969d5/users/6cf7c6fa0089421e92ce4e8d030af7ab/roles/60af5088793e4379b88821a86552a344 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:48:02.137994 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fce8397-ffd7-4bfc-b426-10652dd904b1 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:48:02.142691 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fce8397-ffd7-4bfc-b426-10652dd904b1 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:48:02.142796 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fce8397-ffd7-4bfc-b426-10652dd904b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:48:02.187007 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-0fce8397-ffd7-4bfc-b426-10652dd904b1 None None] MFA Rules not processed for user `6cf7c6fa0089421e92ce4e8d030af7ab`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:48:02.188287 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-0fce8397-ffd7-4bfc-b426-10652dd904b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:48:02.270059 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1438/2877] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:48:02 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:48:02.281728 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd3038f-5de6-4d94-a53f-8285c200b547 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:48:02.283477 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-edd3038f-5de6-4d94-a53f-8285c200b547 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:48:02.294501 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fce8397-ffd7-4bfc-b426-10652dd904b1 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:48:02.295939 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-0fce8397-ffd7-4bfc-b426-10652dd904b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:48:02.364040 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd3038f-5de6-4d94-a53f-8285c200b547 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:48:02.369987 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeef30b-acbe-4e1f-afac-cebab81c1f56 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c1f878b0dc4d63a4f5dcb12fac1dc1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:48:02.370643 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeef30b-acbe-4e1f-afac-cebab81c1f56 service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:48:02.370927 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeef30b-acbe-4e1f-afac-cebab81c1f56 service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:48:02.371963 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fce8397-ffd7-4bfc-b426-10652dd904b1 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:48:02.372272 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeef30b-acbe-4e1f-afac-cebab81c1f56 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:48:02.377268 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0ab2a6-a05c-47be-b38d-beab1cc37bfd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06538aab5f194af7b7c677d6b94aef2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:48:02.378053 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0ab2a6-a05c-47be-b38d-beab1cc37bfd service placement] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:48:02.378339 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0ab2a6-a05c-47be-b38d-beab1cc37bfd service placement] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:48:02.378608 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0ab2a6-a05c-47be-b38d-beab1cc37bfd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:48:02.380723 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-7a0ab2a6-a05c-47be-b38d-beab1cc37bfd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:48:02.396784 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0ab2a6-a05c-47be-b38d-beab1cc37bfd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:48:02.398621 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0ab2a6-a05c-47be-b38d-beab1cc37bfd service placement] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:48:02.409599 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1439/2878] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:48:02 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 17 19:48:02.435669 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeef30b-acbe-4e1f-afac-cebab81c1f56 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:48:02.437457 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeef30b-acbe-4e1f-afac-cebab81c1f56 service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:48:02.446865 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1440/2879] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:48:02 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:48:04.971800 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05182daa-8474-44bd-be36-05cfff064b2a None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:48:04.972268 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05182daa-8474-44bd-be36-05cfff064b2a None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:48:04.972366 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05182daa-8474-44bd-be36-05cfff064b2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:48:04.990272 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-05182daa-8474-44bd-be36-05cfff064b2a None None] MFA Rules not processed for user `02ec33ef01904b21a2727e861735fc65`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:48:04.991470 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-05182daa-8474-44bd-be36-05cfff064b2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:48:05.004502 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1440/2880] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 17 19:48:04 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:48:21.897765 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeef30b-acbe-4e1f-afac-cebab81c1f56 service nova] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:48:21.949823 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeef30b-acbe-4e1f-afac-cebab81c1f56 service nova] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:48:21.953528 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3943c0-7b85-4f3f-aa7a-61902fc82540 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:48:21.954165 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3943c0-7b85-4f3f-aa7a-61902fc82540 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:48:21.954445 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3943c0-7b85-4f3f-aa7a-61902fc82540 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:48:21.954719 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3943c0-7b85-4f3f-aa7a-61902fc82540 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:48:21.956497 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-0d3943c0-7b85-4f3f-aa7a-61902fc82540 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:48:21.965936 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3943c0-7b85-4f3f-aa7a-61902fc82540 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:48:21.967359 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3943c0-7b85-4f3f-aa7a-61902fc82540 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:48:21.975249 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1441/2881] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:48:21 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:48:23.798491 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05182daa-8474-44bd-be36-05cfff064b2a None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:48:23.799911 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-05182daa-8474-44bd-be36-05cfff064b2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:48:23.815739 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05182daa-8474-44bd-be36-05cfff064b2a None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:48:23.820845 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b791b30-c333-4b9b-a301-08250cd5636c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06538aab5f194af7b7c677d6b94aef2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:48:23.821486 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b791b30-c333-4b9b-a301-08250cd5636c service placement] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:48:23.821795 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b791b30-c333-4b9b-a301-08250cd5636c service placement] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:48:23.822071 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b791b30-c333-4b9b-a301-08250cd5636c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:48:23.823897 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-9b791b30-c333-4b9b-a301-08250cd5636c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:48:23.835953 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b791b30-c333-4b9b-a301-08250cd5636c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:48:23.837522 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b791b30-c333-4b9b-a301-08250cd5636c service placement] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:48:23.848055 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1441/2882] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:48:23 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:48:38.447977 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3943c0-7b85-4f3f-aa7a-61902fc82540 service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:48:38.460691 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3943c0-7b85-4f3f-aa7a-61902fc82540 service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:48:38.466055 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d069b5a-0aee-467e-873e-7d4fb5041515 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c1f878b0dc4d63a4f5dcb12fac1dc1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:48:38.466678 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d069b5a-0aee-467e-873e-7d4fb5041515 service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:48:38.466955 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d069b5a-0aee-467e-873e-7d4fb5041515 service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:48:38.467224 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d069b5a-0aee-467e-873e-7d4fb5041515 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:48:38.468970 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-8d069b5a-0aee-467e-873e-7d4fb5041515 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:48:38.480525 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d069b5a-0aee-467e-873e-7d4fb5041515 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:48:38.482382 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d069b5a-0aee-467e-873e-7d4fb5041515 service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:48:38.493102 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1442/2883] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:48:38 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:48:41.326701 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b791b30-c333-4b9b-a301-08250cd5636c service placement] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:48:41.328215 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-9b791b30-c333-4b9b-a301-08250cd5636c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:48:41.339366 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b791b30-c333-4b9b-a301-08250cd5636c service placement] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:48:41.343280 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c6d3d2-f413-4b98-8be2-3274a5c86002 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:48:41.343976 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c6d3d2-f413-4b98-8be2-3274a5c86002 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:48:41.344259 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c6d3d2-f413-4b98-8be2-3274a5c86002 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:48:41.344533 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c6d3d2-f413-4b98-8be2-3274a5c86002 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:48:41.353474 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c6d3d2-f413-4b98-8be2-3274a5c86002 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:48:41.356543 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c6d3d2-f413-4b98-8be2-3274a5c86002 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:48:41.364883 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1442/2884] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:48:41 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:50:39.694404 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d069b5a-0aee-467e-873e-7d4fb5041515 service nova] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:50:39.704068 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d069b5a-0aee-467e-873e-7d4fb5041515 service nova] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:50:39.708591 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e8e059-5afb-45ac-ba35-a734d2756a23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:50:39.710542 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e8e059-5afb-45ac-ba35-a734d2756a23 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:50:39.710542 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e8e059-5afb-45ac-ba35-a734d2756a23 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:50:39.710542 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e8e059-5afb-45ac-ba35-a734d2756a23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:50:39.711889 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-41e8e059-5afb-45ac-ba35-a734d2756a23 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:50:39.727290 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e8e059-5afb-45ac-ba35-a734d2756a23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:50:39.728972 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e8e059-5afb-45ac-ba35-a734d2756a23 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:50:39.740090 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1443/2885] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:50:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:50:40.916808 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c6d3d2-f413-4b98-8be2-3274a5c86002 service neutron] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:50:40.925257 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c6d3d2-f413-4b98-8be2-3274a5c86002 service neutron] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:50:40.928894 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c49de2f-caaf-4208-a64e-f1e1a713286b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:50:40.929538 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c49de2f-caaf-4208-a64e-f1e1a713286b service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:50:40.929818 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c49de2f-caaf-4208-a64e-f1e1a713286b service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:50:40.930090 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c49de2f-caaf-4208-a64e-f1e1a713286b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:50:40.931863 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-7c49de2f-caaf-4208-a64e-f1e1a713286b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:50:40.944953 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c49de2f-caaf-4208-a64e-f1e1a713286b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:50:40.946379 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c49de2f-caaf-4208-a64e-f1e1a713286b service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:50:40.957867 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1443/2886] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:50:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:50:53.955936 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e8e059-5afb-45ac-ba35-a734d2756a23 service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:50:53.964608 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e8e059-5afb-45ac-ba35-a734d2756a23 service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:50:53.969109 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b94bdd4-360f-4e74-a35c-dbc6f651b290 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:50:53.969731 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b94bdd4-360f-4e74-a35c-dbc6f651b290 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:50:53.970007 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b94bdd4-360f-4e74-a35c-dbc6f651b290 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:50:53.970277 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b94bdd4-360f-4e74-a35c-dbc6f651b290 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:50:53.972027 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-8b94bdd4-360f-4e74-a35c-dbc6f651b290 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:50:53.981282 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b94bdd4-360f-4e74-a35c-dbc6f651b290 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:50:53.982944 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b94bdd4-360f-4e74-a35c-dbc6f651b290 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:50:53.990714 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1444/2887] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:50:53 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:50:54.297632 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a799388-db79-4819-ace8-a4740a8ce8f5 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:50:54.298231 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a799388-db79-4819-ace8-a4740a8ce8f5 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:50:54.298231 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a799388-db79-4819-ace8-a4740a8ce8f5 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:50:54.299605 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1444/2888] 10.222.0.58 () {54 vars in 921 bytes} [Tue May 17 19:50:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:50:54.305510 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131fb51f-c0b8-4c7c-8cd0-f3f7880c9245 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:50:54.305797 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131fb51f-c0b8-4c7c-8cd0-f3f7880c9245 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:50:54.306071 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131fb51f-c0b8-4c7c-8cd0-f3f7880c9245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:50:54.353877 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-131fb51f-c0b8-4c7c-8cd0-f3f7880c9245 None None] MFA Rules not processed for user `1ac193d15f48463c98ff1d97770ed794`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:50:54.355077 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-131fb51f-c0b8-4c7c-8cd0-f3f7880c9245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:50:54.362332 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1445/2889] 10.222.0.58 () {58 vars in 1018 bytes} [Tue May 17 19:50:54 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 17 19:50:54.377853 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a799388-db79-4819-ace8-a4740a8ce8f5 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:50:54.394807 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a799388-db79-4819-ace8-a4740a8ce8f5 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:50:54.398352 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e886a0-f859-43ba-936b-0d834fc7e895 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:50:54.398981 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e886a0-f859-43ba-936b-0d834fc7e895 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:50:54.399262 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e886a0-f859-43ba-936b-0d834fc7e895 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:50:54.399583 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e886a0-f859-43ba-936b-0d834fc7e895 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:50:54.406406 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e886a0-f859-43ba-936b-0d834fc7e895 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:50:54.407869 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e886a0-f859-43ba-936b-0d834fc7e895 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:50:54.415581 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1445/2890] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:50:54 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:50:54.497584 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a520ca5-cdbd-4c76-bd56-eecafe6a9035 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:50:54.497870 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a520ca5-cdbd-4c76-bd56-eecafe6a9035 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:50:54.498146 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a520ca5-cdbd-4c76-bd56-eecafe6a9035 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:50:54.499539 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1446/2891] 10.222.0.58 () {54 vars in 921 bytes} [Tue May 17 19:50:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 17 19:50:54.504735 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4be4d64-81aa-4be6-a1fb-a34392ee5a6a None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:50:54.505020 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4be4d64-81aa-4be6-a1fb-a34392ee5a6a None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:50:54.505316 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4be4d64-81aa-4be6-a1fb-a34392ee5a6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:50:54.539149 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-e4be4d64-81aa-4be6-a1fb-a34392ee5a6a None None] MFA Rules not processed for user `65c1f878b0dc4d63a4f5dcb12fac1dc1`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 17 19:50:54.540364 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-e4be4d64-81aa-4be6-a1fb-a34392ee5a6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:50:54.550354 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1446/2892] 10.222.0.58 () {58 vars in 1018 bytes} [Tue May 17 19:50:54 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 17 19:50:54.561101 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a520ca5-cdbd-4c76-bd56-eecafe6a9035 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:50:54.571875 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a520ca5-cdbd-4c76-bd56-eecafe6a9035 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:50:54.575418 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b213d86d-7f42-42fb-859d-8daa9f0fe7c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:50:54.576083 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b213d86d-7f42-42fb-859d-8daa9f0fe7c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:50:54.576362 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b213d86d-7f42-42fb-859d-8daa9f0fe7c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:50:54.576638 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b213d86d-7f42-42fb-859d-8daa9f0fe7c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:50:54.586531 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b213d86d-7f42-42fb-859d-8daa9f0fe7c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:50:54.588001 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b213d86d-7f42-42fb-859d-8daa9f0fe7c9 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:50:54.598396 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1447/2893] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:50:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:51:02.059795 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4be4d64-81aa-4be6-a1fb-a34392ee5a6a None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:51:02.069009 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4be4d64-81aa-4be6-a1fb-a34392ee5a6a None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:51:02.072587 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f236f82-8431-4234-bb55-bc01e020fb5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac193d15f48463c98ff1d97770ed794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:51:02.073213 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f236f82-8431-4234-bb55-bc01e020fb5b service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:51:02.073496 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f236f82-8431-4234-bb55-bc01e020fb5b service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:51:02.073765 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f236f82-8431-4234-bb55-bc01e020fb5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:51:02.076187 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-7f236f82-8431-4234-bb55-bc01e020fb5b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:51:02.085375 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f236f82-8431-4234-bb55-bc01e020fb5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:51:02.090063 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f236f82-8431-4234-bb55-bc01e020fb5b service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:51:02.095616 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1447/2894] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:51:02 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:52:24.176143 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b213d86d-7f42-42fb-859d-8daa9f0fe7c9 service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:52:24.178060 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b213d86d-7f42-42fb-859d-8daa9f0fe7c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:52:24.254550 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b213d86d-7f42-42fb-859d-8daa9f0fe7c9 service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:52:24.260402 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383a3a21-6b16-4cb5-9dbb-0d3136a41db4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c1f878b0dc4d63a4f5dcb12fac1dc1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:52:24.261043 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383a3a21-6b16-4cb5-9dbb-0d3136a41db4 service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:52:24.261317 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383a3a21-6b16-4cb5-9dbb-0d3136a41db4 service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:52:24.261585 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383a3a21-6b16-4cb5-9dbb-0d3136a41db4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:52:24.263688 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-383a3a21-6b16-4cb5-9dbb-0d3136a41db4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:52:24.276143 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383a3a21-6b16-4cb5-9dbb-0d3136a41db4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:52:24.277715 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383a3a21-6b16-4cb5-9dbb-0d3136a41db4 service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:52:24.289138 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1448/2895] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:52:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:52:26.792604 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f236f82-8431-4234-bb55-bc01e020fb5b service neutron] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:52:26.816311 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f236f82-8431-4234-bb55-bc01e020fb5b service neutron] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:52:26.821743 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4bd117-d359-4b8e-ad49-770a6c848572 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c1f878b0dc4d63a4f5dcb12fac1dc1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:52:26.822514 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4bd117-d359-4b8e-ad49-770a6c848572 service nova] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:52:26.822792 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4bd117-d359-4b8e-ad49-770a6c848572 service nova] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:52:26.823059 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4bd117-d359-4b8e-ad49-770a6c848572 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:52:26.824820 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-fc4bd117-d359-4b8e-ad49-770a6c848572 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:52:26.836667 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4bd117-d359-4b8e-ad49-770a6c848572 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:52:26.838629 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4bd117-d359-4b8e-ad49-770a6c848572 service nova] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:52:26.848653 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1448/2896] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:52:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 17 19:52:36.217809 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383a3a21-6b16-4cb5-9dbb-0d3136a41db4 service nova] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:52:36.296960 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383a3a21-6b16-4cb5-9dbb-0d3136a41db4 service nova] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:52:36.301636 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb12e28-ddae-46a5-9800-ea9c6804542d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:52:36.302488 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb12e28-ddae-46a5-9800-ea9c6804542d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:52:36.302764 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb12e28-ddae-46a5-9800-ea9c6804542d admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:52:36.303034 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb12e28-ddae-46a5-9800-ea9c6804542d admin admin] PATH_INFO: `/v3/users/02ec33ef01904b21a2727e861735fc65` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:52:36.304246 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb12e28-ddae-46a5-9800-ea9c6804542d admin admin] RBAC: Authorizing `identity:delete_user(user_id=02ec33ef01904b21a2727e861735fc65)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:52:36.306205 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb12e28-ddae-46a5-9800-ea9c6804542d admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:52:36.441207 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0eb12e28-ddae-46a5-9800-ea9c6804542d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02ec33ef01904b21a2727e861735fc65'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:52:36.441783 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0eb12e28-ddae-46a5-9800-ea9c6804542d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02ec33ef01904b21a2727e861735fc65'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:52:36.453694 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0eb12e28-ddae-46a5-9800-ea9c6804542d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02ec33ef01904b21a2727e861735fc65'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:52:36.514753 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0eb12e28-ddae-46a5-9800-ea9c6804542d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02ec33ef01904b21a2727e861735fc65'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:52:36.521636 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0eb12e28-ddae-46a5-9800-ea9c6804542d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02ec33ef01904b21a2727e861735fc65'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:52:36.527414 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1449/2897] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 17 19:52:36 2022] DELETE /identity/v3/users/02ec33ef01904b21a2727e861735fc65 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:52:36.767954 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4bd117-d359-4b8e-ad49-770a6c848572 service nova] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:52:36.778180 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4bd117-d359-4b8e-ad49-770a6c848572 service nova] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:52:36.834083 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c99977-e150-4714-8883-5026e229db5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:52:36.834939 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c99977-e150-4714-8883-5026e229db5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:52:36.835217 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c99977-e150-4714-8883-5026e229db5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:52:36.835509 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c99977-e150-4714-8883-5026e229db5f admin admin] PATH_INFO: `/v3/projects/757859fa777e4b5dad0105d36b6554d1` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:52:36.836687 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c99977-e150-4714-8883-5026e229db5f admin admin] RBAC: Authorizing `identity:delete_project(project_id=757859fa777e4b5dad0105d36b6554d1)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:52:36.838588 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c99977-e150-4714-8883-5026e229db5f admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:52:36.944867 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-97c99977-e150-4714-8883-5026e229db5f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '757859fa777e4b5dad0105d36b6554d1', 'request_id': 'req-97c99977-e150-4714-8883-5026e229db5f'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:52:36.950956 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-97c99977-e150-4714-8883-5026e229db5f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '757859fa777e4b5dad0105d36b6554d1', 'request_id': 'req-97c99977-e150-4714-8883-5026e229db5f'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:52:36.957227 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-97c99977-e150-4714-8883-5026e229db5f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '757859fa777e4b5dad0105d36b6554d1', 'request_id': 'req-97c99977-e150-4714-8883-5026e229db5f'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:52:37.084760 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-97c99977-e150-4714-8883-5026e229db5f admin admin] The token cache is being invalidate because project 757859fa777e4b5dad0105d36b6554d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 17 19:52:37.084964 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-97c99977-e150-4714-8883-5026e229db5f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:52:37.085642 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-97c99977-e150-4714-8883-5026e229db5f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:52:37.087161 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1449/2898] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 17 19:52:36 2022] DELETE /identity/v3/projects/757859fa777e4b5dad0105d36b6554d1 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:52:37.091070 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb12e28-ddae-46a5-9800-ea9c6804542d admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:52:37.092515 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-0eb12e28-ddae-46a5-9800-ea9c6804542d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:52:37.105085 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb12e28-ddae-46a5-9800-ea9c6804542d admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:52:37.162103 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d2abac-6645-454d-8629-94eaee2b312c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:52:37.162992 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d2abac-6645-454d-8629-94eaee2b312c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:52:37.163272 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d2abac-6645-454d-8629-94eaee2b312c admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:52:37.163574 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d2abac-6645-454d-8629-94eaee2b312c admin admin] PATH_INFO: `/v3/users/6cf7c6fa0089421e92ce4e8d030af7ab` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:52:37.164767 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d2abac-6645-454d-8629-94eaee2b312c admin admin] RBAC: Authorizing `identity:delete_user(user_id=6cf7c6fa0089421e92ce4e8d030af7ab)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:52:37.166691 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d2abac-6645-454d-8629-94eaee2b312c admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:52:37.327469 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d6d2abac-6645-454d-8629-94eaee2b312c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6cf7c6fa0089421e92ce4e8d030af7ab'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:52:37.328185 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d6d2abac-6645-454d-8629-94eaee2b312c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6cf7c6fa0089421e92ce4e8d030af7ab'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:52:37.339316 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d6d2abac-6645-454d-8629-94eaee2b312c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6cf7c6fa0089421e92ce4e8d030af7ab'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:52:37.414027 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d6d2abac-6645-454d-8629-94eaee2b312c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6cf7c6fa0089421e92ce4e8d030af7ab'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:52:37.421438 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d6d2abac-6645-454d-8629-94eaee2b312c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6cf7c6fa0089421e92ce4e8d030af7ab'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:52:37.427308 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1450/2899] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 17 19:52:37 2022] DELETE /identity/v3/users/6cf7c6fa0089421e92ce4e8d030af7ab => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:52:37.724791 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c99977-e150-4714-8883-5026e229db5f admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:52:37.727921 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-97c99977-e150-4714-8883-5026e229db5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:52:37.740386 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c99977-e150-4714-8883-5026e229db5f admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:52:37.794394 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b4697e-d4ee-4317-aa19-98a41d3c7387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2441e330cd41aaa04d125461331f3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bd98f7df3a477a8bf75754083485f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:52:37.795916 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b4697e-d4ee-4317-aa19-98a41d3c7387 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:52:37.796200 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b4697e-d4ee-4317-aa19-98a41d3c7387 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:52:37.796474 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b4697e-d4ee-4317-aa19-98a41d3c7387 admin admin] PATH_INFO: `/v3/projects/c4e2d65fdc32405ba21ff6081ce969d5` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:52:37.797657 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b4697e-d4ee-4317-aa19-98a41d3c7387 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4e2d65fdc32405ba21ff6081ce969d5)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:52:37.799598 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b4697e-d4ee-4317-aa19-98a41d3c7387 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:52:37.908264 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-51b4697e-d4ee-4317-aa19-98a41d3c7387 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4e2d65fdc32405ba21ff6081ce969d5', 'request_id': 'req-51b4697e-d4ee-4317-aa19-98a41d3c7387'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:52:37.914075 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-51b4697e-d4ee-4317-aa19-98a41d3c7387 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4e2d65fdc32405ba21ff6081ce969d5', 'request_id': 'req-51b4697e-d4ee-4317-aa19-98a41d3c7387'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:52:37.919554 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-51b4697e-d4ee-4317-aa19-98a41d3c7387 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4e2d65fdc32405ba21ff6081ce969d5', 'request_id': 'req-51b4697e-d4ee-4317-aa19-98a41d3c7387'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:52:38.017419 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-51b4697e-d4ee-4317-aa19-98a41d3c7387 admin admin] The token cache is being invalidate because project c4e2d65fdc32405ba21ff6081ce969d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 17 19:52:38.017640 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-51b4697e-d4ee-4317-aa19-98a41d3c7387 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:52:38.018248 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-51b4697e-d4ee-4317-aa19-98a41d3c7387 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 17 19:52:38.019852 dv-cinder-840678-10-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1450/2900] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 17 19:52:37 2022] DELETE /identity/v3/projects/c4e2d65fdc32405ba21ff6081ce969d5 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 17 19:53:06.284551 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d2abac-6645-454d-8629-94eaee2b312c admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 17 19:53:06.285953 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d6d2abac-6645-454d-8629-94eaee2b312c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:53:06.372428 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d2abac-6645-454d-8629-94eaee2b312c admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 17 19:53:06.428935 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82b04ed-9473-42ac-87f0-7e3142a985af service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06538aab5f194af7b7c677d6b94aef2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a9d3a0631284c9fb113266036993e2d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 17 19:53:06.429572 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82b04ed-9473-42ac-87f0-7e3142a985af service placement] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 17 19:53:06.429852 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82b04ed-9473-42ac-87f0-7e3142a985af service placement] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 17 19:53:06.430124 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82b04ed-9473-42ac-87f0-7e3142a985af service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 17 19:53:06.431915 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-e82b04ed-9473-42ac-87f0-7e3142a985af service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 17 19:53:06.443446 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82b04ed-9473-42ac-87f0-7e3142a985af service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 17 19:53:06.445058 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82b04ed-9473-42ac-87f0-7e3142a985af service placement] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 17 19:53:06.455286 dv-cinder-840678-10-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1451/2901] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 17 19:53:06 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0)